-- Logs begin at Fri 2022-07-22 10:32:21 UTC, end at Fri 2022-07-22 11:33:59 UTC. -- Jul 22 10:59:38.293394 dv-cinder-848900-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 22 10:59:38.302762 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 22 10:59:38 2022] *** Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: nodename: dv-cinder-848900-4-iscsi Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: machine: x86_64 Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: clock source: unix Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: pcre jit disabled Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: detected number of CPU cores: 4 Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: current working directory: / Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: detected binary path: /usr/bin/uwsgi-core Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: your processes number limit is 47680 Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: your memory page size is 4096 bytes Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: detected max file descriptor number: 2048 Jul 22 10:59:38.304756 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: lock engine: pthread robust mutexes Jul 22 10:59:38.306333 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: thunder lock: enabled Jul 22 10:59:38.306333 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 22 10:59:38.306333 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 22 10:59:38.351548 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: Python main interpreter initialized at 0x555d2f04a9c0 Jul 22 10:59:38.351548 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: python threads support enabled Jul 22 10:59:38.351548 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: your server socket listen backlog is limited to 100 connections Jul 22 10:59:38.351548 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: your mercy for graceful operations on workers is 90 seconds Jul 22 10:59:38.352006 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: mapped 403077 bytes (393 KB) for 2 cores Jul 22 10:59:38.352006 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: *** Operational MODE: preforking *** Jul 22 10:59:38.352279 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: *** uWSGI is running in multiple interpreter mode *** Jul 22 10:59:38.352279 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: spawned uWSGI master process (pid: 80378) Jul 22 10:59:38.352494 dv-cinder-848900-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 22 10:59:38.354106 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: spawned uWSGI worker 1 (pid: 80393, cores: 1) Jul 22 10:59:38.354106 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: spawned uWSGI worker 2 (pid: 80394, cores: 1) Jul 22 10:59:38.354106 dv-cinder-848900-4-iscsi devstack@keystone.service[80378]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 22 10:59:43.903156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 22 10:59:43.903156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes Jul 22 10:59:43.903953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 22 10:59:43.903953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes Jul 22 10:59:43.904066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 22 10:59:43.904066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: from cryptography.utils import int_from_bytes Jul 22 10:59:43.904493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 22 10:59:43.904493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: from cryptography.utils import int_from_bytes Jul 22 10:59:43.953748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 22 10:59:43.954062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 22 10:59:43.954166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 22 10:59:43.954258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] command line args: [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 22 10:59:43.954342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 22 10:59:43.954492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 22 10:59:43.954577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] command line args: [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 22 10:59:43.954703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ================================================================================ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 22 10:59:43.954788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 22 10:59:43.954985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ================================================================================ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 22 10:59:43.955073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.955288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] admin_token = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.955428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_dir = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.955551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.955684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config_dir = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.955814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_source = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.955960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.956099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] debug = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.956238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config_source = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.956435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.956773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] debug = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.956773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.957143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.957312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.957478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.957603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.957701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.957795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.957889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.957983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_config_append = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_config_append = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_dir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_dir = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.958993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.959085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.959177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.959305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.959420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.959523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.959650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.959726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.959837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.959951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960038 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960667 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.960938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.961055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.961173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.961371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.961478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.961601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.961720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.961812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.962008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.962095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.962289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.962412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.962806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.962964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.963063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] publish_errors = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.963153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] publish_errors = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.963286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.963384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.963475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.963579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.963676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.963781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.963886 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.964007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.964105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.964200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.964332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] standard_threads = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.964522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] standard_threads = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.964669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.964751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.964831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.964903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_journal = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_journal = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_json = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_json = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_stderr = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_stderr = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_syslog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_syslog = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.965968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.966042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 22 10:59:43.966163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.966245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.966317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.966437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.966555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.966633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.966797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.966878 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.967063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.967151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.967318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.967398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.967574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.967661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.967814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.967891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.968064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.968144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.968315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.968401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.968567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.968644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.968804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.968889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.969053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.969131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.969291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.969371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.970062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.970149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.970323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.970409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.970573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.970649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.970837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.970918 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.971118 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.971202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.971388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.971467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.971632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.971712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.971893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.971971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.972156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.972231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.972412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.972498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.972664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.972743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.972898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.972980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.973131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.973208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.973364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.973448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.973602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.973679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.973853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.973936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.974104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.974182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.974337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.974413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.974570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.974655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.974820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.974896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.975070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.975146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.975340 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.975433 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.975594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.975670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.975829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.975905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.976070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.976221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.976295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.976369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.976526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.976603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.976759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.976842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.976989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.977065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.977222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.977302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.977462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.977545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.977711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.977788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.977944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.978027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.978185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.978270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.978417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.978493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.978654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.978730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.979094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.979223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.979366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.979466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.979562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.979663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.979790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.979903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.980864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.981024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.981111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.981232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.981312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.981439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.981515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.981653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.981785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.981929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.982104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.982220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.982325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.982416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.982515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.982606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.982706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.982833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.982936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.983109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.983222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.983386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.983498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.983609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.983715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.983854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.984063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.984172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.984320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.984397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.984458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.984558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.984637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.984780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.984897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.985016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.985131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.985329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.985940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.986067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.986145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.986287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.986368 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.986526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.986607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.986765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.986844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.987004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.987085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.987243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.987389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.987542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.987623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.987787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.987868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.988033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.988114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.988305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.988384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.988498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.988573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.988732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.988813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.988973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.989064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.989239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.989326 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.989451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.989532 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.989647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.989723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.989879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.989961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.990119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.990200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.990355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.990436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.990611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.990839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.990937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.991101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.991252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.991619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.991882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.991973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.992154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.992400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.992487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.992635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.external = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.992863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.992951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.993167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.993339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.993427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.993746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.password = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.993891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.993997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.token = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.994096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.994171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.external = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.994400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.994575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.994742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.994824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.995061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.995150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.995411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.995548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.995701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.995822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.995964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.password = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.996080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.996339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.996418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.token = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.996599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.996788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.996869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.997088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.997222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.997335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.997617 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.997704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.997890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.998072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.998152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.998389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.998535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.998688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.998945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.999023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.999215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.999425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.999508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.999745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.999826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:43.999990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.000237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.000315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.000491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.000630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.000777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.001019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.001096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.001276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.001453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.001599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.001777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.001911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.002025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.002138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.002275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.002410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.002506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.002645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.002759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.002890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.003019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.003156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.003316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.003471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.003606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.003729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.003972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.004420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.004669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.004924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.005187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.005454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.005698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.005940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.006181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.006437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.006696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.006964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.007438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.007664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.007931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.008014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.008242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.008451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.008530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.008781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.008922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.009043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.009269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.009360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.009521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.009633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.009777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.009884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.010024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.010146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.010273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.010399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.010530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.010665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.010782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.010913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.011182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.011328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.011455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.011586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.011711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.011850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.011987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.012111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.012191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.012358 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.012437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.012618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.012705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.012877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.012985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.013119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.013337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.013460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.013580 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.013828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.013905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.014085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.014346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.014423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.014584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.014802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.014881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.015041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.015312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.015396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.015565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.015787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.015866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.016073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.016263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.016343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.016634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.016713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.016905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.016972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.017149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.017217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.017383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.017452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.017628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.017697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.017863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.018065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.018166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.018398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.018477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.018643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.018815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.018894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.019071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.019151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.019345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.019429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.019608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.019730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.019898 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.019987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.020115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.020195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.020377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.020467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.020630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.020716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.020884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.020952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.021130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.021200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.021370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.021440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.021615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.021682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.021856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.021924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.022097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.022166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.022331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.022402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.022744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.022880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.022880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.023120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.023243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.023448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.023560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.023689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.023804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.023916 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.024028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.024145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.024264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.024387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.024500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.024611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.024720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.024838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.024948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.025068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.025179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.025289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.025399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.025510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.025510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.025729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.025841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.025950 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026803 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026878 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.026959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.027058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.027145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.027325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.027411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.027570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.027659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.027814 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.027914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.028059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.028139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.028305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.028390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.028553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.028632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.028802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.028881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.029037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.029159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.029292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.029399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.029529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.029645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.029755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.029884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.029992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.030073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.030231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.030310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.030480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.030568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.030721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.030799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.030963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.031053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.031213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.031318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.031471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.031550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.031720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.031798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.031960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.032578 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.032757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.032868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.033015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.033127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.033273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.033481 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.033563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.033635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.033738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.033817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.033977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.034065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.034221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.034300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.034474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.034563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.034721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.034808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.034960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.035040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.035212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.035429 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.035536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.035678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.035789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.035935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.036046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.036183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.036303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.036424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.036541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.036647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.036727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.036902 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.036989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.037135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.037213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.037389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.037467 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.037641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.037728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.037879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.037957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.038118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.038196 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.038364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.038450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.038610 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.038688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.038859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.038937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.039108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.039195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.039430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.039521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.039631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.039795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.039937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.040098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.040238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.040376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.040485 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.040562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.040636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.040771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.040855 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.041011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.041092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.041266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.041346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.041513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.041592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.041763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.041842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.042011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.042089 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.042247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.042330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.042500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.042578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.042747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.042826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.042989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.043068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.043225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.043327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.043487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.043589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.043723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.043803 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.043971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.044049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.044229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.044316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.044483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.044562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.044724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.044812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.044963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.045042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.045196 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.045283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.045435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.045514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.045675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.045754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.045926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.046014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.046165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.046244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.046410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.046487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.046660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.046745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.046904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.046983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.047149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.047226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.047419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.047512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.047730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.048296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.048460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.048539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.048701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.048788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.048941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.049019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.049191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.049293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.049434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.049522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.049596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 22 10:59:44.049668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.049897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.050206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.050476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.050726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.050975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.051217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.051488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.051726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.051966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.052206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.052445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.052692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.052933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.053180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.053419 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.053671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.053908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.054157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.054391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.054636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.054873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 22 10:59:44.055046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 22 10:59:44.081174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.081385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.081472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.081549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.081623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.081800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.090361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.090540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.120367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.120600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.120966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.121211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.266524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.266895 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.318063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.318322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.319474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.319725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.386235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.386433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.386509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.386660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.386751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.386876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.386960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.387093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.387195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.387337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.387436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.387560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.414299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.414514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.414923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.415310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.415397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.415543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.415767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.415865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.415986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.416261 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.416492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.416715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.416934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.417155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.427699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.428067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.440948 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.441160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.441393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.441611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.441732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.441820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.441976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.442069 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.442208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.442291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.442434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.442657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.442878 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.443099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 22 10:59:44.460116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.460329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 22 10:59:44.460613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.460767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.460888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 22 10:59:44.461011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 22 10:59:44.461135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.461258 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.461377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.461519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 22 10:59:44.461632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.461753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.461876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.461993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.462123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.462362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.473513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.473799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.474068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.474148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.474349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.474432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.474622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.474912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.482458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.483062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.485768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.486049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.486333 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.486412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.486668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.486748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.486956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.487035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.487274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.487557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.487658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.487833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.487912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.488104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.488182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.488379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.488459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.488646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.488725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.488921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.489199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.489474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.515350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.517311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.517525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.517834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.518916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.519204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.523059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.523419 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.523651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.524542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.524826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.525114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.530923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.531208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.533000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.533293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.536394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.536682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.536964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.537246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.537522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.537798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.538062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.538152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.538326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.538415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.538619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.538905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.539187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.539495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.539771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.540045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.557084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.557385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.558706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.559001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.562063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.563681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.565791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.566087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.567342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.567650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.571114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.572702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.574758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.576365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.578527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.579343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.579650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.579934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.580227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.580500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.580721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.580978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.581328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.581638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.581955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.582250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.600091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.600509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.600781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.601053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.602139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.602553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.602840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.603110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.609041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.610819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.611092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.611182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.612899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.613181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.614160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.614442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.614722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.615002 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.615371 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.616502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.616819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.617388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.617680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.617968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.629011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.629298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.629582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.629870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.630151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.630437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.630712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.631618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.631921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.632212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.632493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.632785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.633080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.633371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.650702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.653440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.654009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.656777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.657882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.659513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.660678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.661148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.661428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.661716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.662302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.663976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.664261 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.664552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.669244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.670873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.672043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.673722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.674133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.674428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.674771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.675057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.676986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.677289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.677581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.677863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.684325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.684620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.684900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.687175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.687500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.687786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.693312 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.693598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.693878 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.694147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.694416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.694697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.694968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.695237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.695532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.695797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.696064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.696242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 22 10:59:44.696327 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.696530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.696620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.696812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.696897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.697090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.697365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.697647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.697931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.698203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.698489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.698763 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.699043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.699331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.699601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.699873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 22 10:59:44.735215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555d2f04a9c0 pid: 80394 (default app) Jul 22 10:59:44.738619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555d2f04a9c0 pid: 80393 (default app) Jul 22 10:59:44.738728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d10634-0f6f-43b4-a1fd-df1fb158ada1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:44.738972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d10634-0f6f-43b4-a1fd-df1fb158ada1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:44.739277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d10634-0f6f-43b4-a1fd-df1fb158ada1 None None] PATH_INFO: `/v3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:44.743800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Fri Jul 22 10:59:44 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 10:59:52.679016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34e108-9d23-4a5f-9bc2-ea11f7c8d135 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:52.679628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34e108-9d23-4a5f-9bc2-ea11f7c8d135 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:52.679628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34e108-9d23-4a5f-9bc2-ea11f7c8d135 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:52.684013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 10:59:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 10:59:52.689842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:52.690358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:52.690459 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:52.759725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: INFO dbcounter [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] Registered counter for database keystone Jul 22 10:59:52.782460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG oslo_db.sqlalchemy.engines [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80394) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 22 10:59:52.788974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writer thread running {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 22 10:59:52.973621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80394) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 22 10:59:52.974250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80394) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 22 10:59:53.030988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 10:59:53.032752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 10:59:53.162305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 10:59:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 22 10:59:53.326901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34e108-9d23-4a5f-9bc2-ea11f7c8d135 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:53.398556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: INFO dbcounter [None req-0f34e108-9d23-4a5f-9bc2-ea11f7c8d135 None None] Registered counter for database keystone Jul 22 10:59:53.418189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG oslo_db.sqlalchemy.engines [None req-0f34e108-9d23-4a5f-9bc2-ea11f7c8d135 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80393) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 22 10:59:53.425095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writer thread running {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 22 10:59:53.545039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34e108-9d23-4a5f-9bc2-ea11f7c8d135 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:53.550019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10ac3e-62f4-45d5-b739-87114046f4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:53.550692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10ac3e-62f4-45d5-b739-87114046f4aa admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:53.550982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10ac3e-62f4-45d5-b739-87114046f4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:53.551292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10ac3e-62f4-45d5-b739-87114046f4aa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:53.560111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10ac3e-62f4-45d5-b739-87114046f4aa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:53.612160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ac10ac3e-62f4-45d5-b739-87114046f4aa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 22 10:59:53.612508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10ac3e-62f4-45d5-b739-87114046f4aa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:53.623229 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Fri Jul 22 10:59:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 22 10:59:53.629966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:53.645174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f2671-9af4-4215-9b4a-84d47fbad35b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:53.649743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47cdb4-c873-45e0-a365-04f527f12cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:53.650937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47cdb4-c873-45e0-a365-04f527f12cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:53.651227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47cdb4-c873-45e0-a365-04f527f12cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:53.651544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47cdb4-c873-45e0-a365-04f527f12cbb admin admin] PATH_INFO: `/v3/projects/a25f160b71984d6fbacad5d6c1f3248c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:53.655816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47cdb4-c873-45e0-a365-04f527f12cbb admin admin] RBAC: Authorizing `identity:get_project(project_id=a25f160b71984d6fbacad5d6c1f3248c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:53.707919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4e47cdb4-c873-45e0-a365-04f527f12cbb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 22 10:59:53.708594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47cdb4-c873-45e0-a365-04f527f12cbb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:53.713187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Fri Jul 22 10:59:53 2022] GET /identity/v3/projects/a25f160b71984d6fbacad5d6c1f3248c => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 10:59:55.799601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b24bb3-e375-4f0c-b8e0-ded2f515c073 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:55.800174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b24bb3-e375-4f0c-b8e0-ded2f515c073 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:55.800174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b24bb3-e375-4f0c-b8e0-ded2f515c073 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:55.805931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 10:59:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 10:59:55.812948 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab901a3b-449d-420c-a6be-6337ce5fe697 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:55.813459 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab901a3b-449d-420c-a6be-6337ce5fe697 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:55.813551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab901a3b-449d-420c-a6be-6337ce5fe697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:55.832961 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ab901a3b-449d-420c-a6be-6337ce5fe697 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 10:59:55.834541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ab901a3b-449d-420c-a6be-6337ce5fe697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 10:59:55.842518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 10:59:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 22 10:59:55.865635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b24bb3-e375-4f0c-b8e0-ded2f515c073 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:55.878089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b24bb3-e375-4f0c-b8e0-ded2f515c073 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:55.882653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ddd8a-8d1d-4c2e-bc57-7e24497286a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:55.883365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ddd8a-8d1d-4c2e-bc57-7e24497286a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:55.883645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ddd8a-8d1d-4c2e-bc57-7e24497286a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:55.883927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ddd8a-8d1d-4c2e-bc57-7e24497286a8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:55.893004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ddd8a-8d1d-4c2e-bc57-7e24497286a8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:55.894802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ddd8a-8d1d-4c2e-bc57-7e24497286a8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:55.904262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Fri Jul 22 10:59:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 10:59:55.910818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab901a3b-449d-420c-a6be-6337ce5fe697 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:55.920148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab901a3b-449d-420c-a6be-6337ce5fe697 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:55.924756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c51a0b-436f-46e0-afef-abd6c6ca4883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:55.925640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c51a0b-436f-46e0-afef-abd6c6ca4883 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:55.925929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c51a0b-436f-46e0-afef-abd6c6ca4883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:55.926212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c51a0b-436f-46e0-afef-abd6c6ca4883 admin admin] PATH_INFO: `/v3/users/9a8c4f38edad4724ac733753692dda22` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:55.927665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c51a0b-436f-46e0-afef-abd6c6ca4883 admin admin] RBAC: Authorizing `identity:get_user(user_id=9a8c4f38edad4724ac733753692dda22)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:55.929514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c51a0b-436f-46e0-afef-abd6c6ca4883 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:55.931143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Fri Jul 22 10:59:55 2022] GET /identity/v3/users/9a8c4f38edad4724ac733753692dda22 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 10:59:58.067975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39df98-5892-45c0-b219-fea6b108fc51 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.068517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39df98-5892-45c0-b219-fea6b108fc51 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.068517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39df98-5892-45c0-b219-fea6b108fc51 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.069908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 10:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 10:59:58.075631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b57481-80e2-4e0d-90b2-bd20c316db8f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.076123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b57481-80e2-4e0d-90b2-bd20c316db8f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.076231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b57481-80e2-4e0d-90b2-bd20c316db8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.094535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f9b57481-80e2-4e0d-90b2-bd20c316db8f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 10:59:58.095012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c3420-7ce9-4533-8fcc-80da2cb82c9f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.095352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c3420-7ce9-4533-8fcc-80da2cb82c9f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.095680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c3420-7ce9-4533-8fcc-80da2cb82c9f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.095779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f9b57481-80e2-4e0d-90b2-bd20c316db8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 10:59:58.097055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 10:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 10:59:58.102979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656de04e-2b33-4713-80ff-3377b64ef5d9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.103349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656de04e-2b33-4713-80ff-3377b64ef5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.103582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656de04e-2b33-4713-80ff-3377b64ef5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.160758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG passlib.handlers.bcrypt [None req-656de04e-2b33-4713-80ff-3377b64ef5d9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80393) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 22 10:59:58.161181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG passlib.handlers.bcrypt [None req-656de04e-2b33-4713-80ff-3377b64ef5d9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80393) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 22 10:59:58.189574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 10:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 10:59:58.209170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-656de04e-2b33-4713-80ff-3377b64ef5d9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 10:59:58.210777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-656de04e-2b33-4713-80ff-3377b64ef5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 10:59:58.213483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b57481-80e2-4e0d-90b2-bd20c316db8f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:58.238996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 10:59:58.324494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b57481-80e2-4e0d-90b2-bd20c316db8f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:58.329405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 10:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 10:59:58.351632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656de04e-2b33-4713-80ff-3377b64ef5d9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:58.360857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d966c-6a95-4758-8b20-67da37d8dea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:58.361682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d966c-6a95-4758-8b20-67da37d8dea6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.361972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d966c-6a95-4758-8b20-67da37d8dea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.362262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d966c-6a95-4758-8b20-67da37d8dea6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.375620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d966c-6a95-4758-8b20-67da37d8dea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-074d966c-6a95-4758-8b20-67da37d8dea6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 10:59:58.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 10:59:58.388233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 10:59:58.389255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 10:59:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 10:59:58.395160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d966c-6a95-4758-8b20-67da37d8dea6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:58.458053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656de04e-2b33-4713-80ff-3377b64ef5d9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:58.460425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 10:59:58.480477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 10:59:58.499249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3495d9-b05e-4db8-96ab-c0ecb8281167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:58.499557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d966c-6a95-4758-8b20-67da37d8dea6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:58.500937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3495d9-b05e-4db8-96ab-c0ecb8281167 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.501169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3495d9-b05e-4db8-96ab-c0ecb8281167 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.501455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3495d9-b05e-4db8-96ab-c0ecb8281167 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.534812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d888f61-a1c5-4890-bb81-696e7c24f16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:58.535515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d888f61-a1c5-4890-bb81-696e7c24f16b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.535805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d888f61-a1c5-4890-bb81-696e7c24f16b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.536086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d888f61-a1c5-4890-bb81-696e7c24f16b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.536847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d888f61-a1c5-4890-bb81-696e7c24f16b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 10:59:58.537155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d888f61-a1c5-4890-bb81-696e7c24f16b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:58.538560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d888f61-a1c5-4890-bb81-696e7c24f16b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:58.552023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 10:59:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 10:59:58.557766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d888f61-a1c5-4890-bb81-696e7c24f16b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:58.575316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3495d9-b05e-4db8-96ab-c0ecb8281167 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:58.576782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3495d9-b05e-4db8-96ab-c0ecb8281167 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:58.661669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d888f61-a1c5-4890-bb81-696e7c24f16b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:58.677979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 10:59:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 22 10:59:58.684029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3495d9-b05e-4db8-96ab-c0ecb8281167 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:58.686989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 10:59:58.697976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8234b-e34f-4468-bd96-cb7e2ccad8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:58.698871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8234b-e34f-4468-bd96-cb7e2ccad8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.699164 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8234b-e34f-4468-bd96-cb7e2ccad8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.699493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8234b-e34f-4468-bd96-cb7e2ccad8f4 None admin] PATH_INFO: `/v3/users/9a8c4f38edad4724ac733753692dda22` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.700729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8234b-e34f-4468-bd96-cb7e2ccad8f4 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8c4f38edad4724ac733753692dda22)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:58.702220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8234b-e34f-4468-bd96-cb7e2ccad8f4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:58.703987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 10:59:58 2022] GET /identity/v3/users/9a8c4f38edad4724ac733753692dda22 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 10:59:58.709437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8234b-e34f-4468-bd96-cb7e2ccad8f4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:58.724280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 10:59:58.789640 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3495d9-b05e-4db8-96ab-c0ecb8281167 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:58.821097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8234b-e34f-4468-bd96-cb7e2ccad8f4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:58.826755 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c8d4-825f-46c5-a699-71a63b63edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:58.827525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c8d4-825f-46c5-a699-71a63b63edd8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.827829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c8d4-825f-46c5-a699-71a63b63edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.828114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c8d4-825f-46c5-a699-71a63b63edd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.846738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c8d4-825f-46c5-a699-71a63b63edd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:58.848275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c8d4-825f-46c5-a699-71a63b63edd8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:58.856848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433dc93-8c9b-4c59-a126-7fe8ea968542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:58.857665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433dc93-8c9b-4c59-a126-7fe8ea968542 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:58.857960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433dc93-8c9b-4c59-a126-7fe8ea968542 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:58.858245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433dc93-8c9b-4c59-a126-7fe8ea968542 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:58.860096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433dc93-8c9b-4c59-a126-7fe8ea968542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:58.861568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433dc93-8c9b-4c59-a126-7fe8ea968542 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:58.863205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 10:59:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 10:59:58.869237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433dc93-8c9b-4c59-a126-7fe8ea968542 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-73f5c8d4-825f-46c5-a699-71a63b63edd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 10:59:58.871541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 10:59:58.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 10:59:58.874580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 10:59:58.874580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 10:59:58.874580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 10:59:58.874580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 10:59:58.874580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 10:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 10:59:58.879467 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c8d4-825f-46c5-a699-71a63b63edd8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 10:59:58.938279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 10:59:58.953230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 10:59:58.975906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433dc93-8c9b-4c59-a126-7fe8ea968542 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:58.985207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c8d4-825f-46c5-a699-71a63b63edd8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 10:59:59.013326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db54e5-e514-4512-854d-e1974392ac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:59.014003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db54e5-e514-4512-854d-e1974392ac3d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:59.014290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db54e5-e514-4512-854d-e1974392ac3d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:59.014570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db54e5-e514-4512-854d-e1974392ac3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:59.015477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db54e5-e514-4512-854d-e1974392ac3d None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.domain.id=default, user.id=9a8c4f38edad4724ac733753692dda22) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 10:59:59.015793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db54e5-e514-4512-854d-e1974392ac3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:59.017257 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db54e5-e514-4512-854d-e1974392ac3d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:59.020160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faeac71-91c7-4443-b903-1a5bf1932685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 10:59:59.020823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeac71-91c7-4443-b903-1a5bf1932685 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 10:59:59.021110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeac71-91c7-4443-b903-1a5bf1932685 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 10:59:59.021389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeac71-91c7-4443-b903-1a5bf1932685 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 10:59:59.022116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faeac71-91c7-4443-b903-1a5bf1932685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 10:59:59.022422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faeac71-91c7-4443-b903-1a5bf1932685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 10:59:59.023914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faeac71-91c7-4443-b903-1a5bf1932685 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 10:59:59.024491 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Fri Jul 22 10:59:58 2022] GET /identity/v3/role_assignments?user.id=9a8c4f38edad4724ac733753692dda22&scope.domain.id=default&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 10:59:59.044627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 10:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:01.467776 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28730b61-0d4d-4e7d-aeab-e7b4d4b9c492 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:01.468904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28730b61-0d4d-4e7d-aeab-e7b4d4b9c492 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:01.468904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28730b61-0d4d-4e7d-aeab-e7b4d4b9c492 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:01.470411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:01.475583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1abf9-37f3-43a9-843c-6cc9e16c4cbb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:01.476007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1abf9-37f3-43a9-843c-6cc9e16c4cbb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:01.476197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1abf9-37f3-43a9-843c-6cc9e16c4cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:01.500643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c2c1abf9-37f3-43a9-843c-6cc9e16c4cbb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:01.501910 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c2c1abf9-37f3-43a9-843c-6cc9e16c4cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:01.585804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 11/23] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:01.628781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28730b61-0d4d-4e7d-aeab-e7b4d4b9c492 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:01.763296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd6ce5-d56f-4bb3-ade3-3a556cb5ee6c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:01.763492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd6ce5-d56f-4bb3-ade3-3a556cb5ee6c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:01.763865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd6ce5-d56f-4bb3-ade3-3a556cb5ee6c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:01.766698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:01.774428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9862d3d-9c76-439f-a7db-de2b06f0f3e3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:01.775109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9862d3d-9c76-439f-a7db-de2b06f0f3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:01.775747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9862d3d-9c76-439f-a7db-de2b06f0f3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:01.814782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28730b61-0d4d-4e7d-aeab-e7b4d4b9c492 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:01.828913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f9862d3d-9c76-439f-a7db-de2b06f0f3e3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:01.830138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f9862d3d-9c76-439f-a7db-de2b06f0f3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:01.915020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da485bc-2332-44bf-bbb5-82e7c1c1ab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:01.915874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da485bc-2332-44bf-bbb5-82e7c1c1ab07 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:01.916173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da485bc-2332-44bf-bbb5-82e7c1c1ab07 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:01.916464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da485bc-2332-44bf-bbb5-82e7c1c1ab07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:01.939654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:01.944842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da485bc-2332-44bf-bbb5-82e7c1c1ab07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3da485bc-2332-44bf-bbb5-82e7c1c1ab07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:01.964564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:01.966099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:01.968073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 13/25] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:01.980506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da485bc-2332-44bf-bbb5-82e7c1c1ab07 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:01.990210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:02.013094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 13/26] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:02.016649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb7063-a4e3-405e-a0cc-ca01d5823cfe None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.017141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb7063-a4e3-405e-a0cc-ca01d5823cfe None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.017668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb7063-a4e3-405e-a0cc-ca01d5823cfe None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.019514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 14/27] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:02.024728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe27da-7491-4d80-b463-faeecc5deb70 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.025353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe27da-7491-4d80-b463-faeecc5deb70 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.025819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe27da-7491-4d80-b463-faeecc5deb70 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.027306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 15/28] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:02.028620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acad78c-a27e-49cd-92bb-cd0f19036bcb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.029169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acad78c-a27e-49cd-92bb-cd0f19036bcb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.029657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acad78c-a27e-49cd-92bb-cd0f19036bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.049515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1acad78c-a27e-49cd-92bb-cd0f19036bcb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:02.051050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1acad78c-a27e-49cd-92bb-cd0f19036bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:02.121952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da485bc-2332-44bf-bbb5-82e7c1c1ab07 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:02.147303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 16/29] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:02.154412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c5e30-0ed8-4292-a1af-4da4d1d9ca64 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.154412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c5e30-0ed8-4292-a1af-4da4d1d9ca64 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.154412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c5e30-0ed8-4292-a1af-4da4d1d9ca64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.183172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7588b-e697-4ea3-abe2-6cb5d41f1497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:02.183884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7588b-e697-4ea3-abe2-6cb5d41f1497 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.184179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7588b-e697-4ea3-abe2-6cb5d41f1497 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.184466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7588b-e697-4ea3-abe2-6cb5d41f1497 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.185237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7588b-e697-4ea3-abe2-6cb5d41f1497 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:02.185549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7588b-e697-4ea3-abe2-6cb5d41f1497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:02.187496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7588b-e697-4ea3-abe2-6cb5d41f1497 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:02.189033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-333c5e30-0ed8-4292-a1af-4da4d1d9ca64 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:02.190982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-333c5e30-0ed8-4292-a1af-4da4d1d9ca64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:02.196685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 14/30] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:02.197561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7588b-e697-4ea3-abe2-6cb5d41f1497 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:02.272694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:02.285274 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 17/31] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:02.286465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c5e30-0ed8-4292-a1af-4da4d1d9ca64 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:02.333762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7588b-e697-4ea3-abe2-6cb5d41f1497 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:02.348594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:02.372589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899f841-b683-40a9-ab4c-f12439bc53ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:02.373657 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899f841-b683-40a9-ab4c-f12439bc53ef None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.374138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899f841-b683-40a9-ab4c-f12439bc53ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.375229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899f841-b683-40a9-ab4c-f12439bc53ef None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.376223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899f841-b683-40a9-ab4c-f12439bc53ef None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:02.377894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899f841-b683-40a9-ab4c-f12439bc53ef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:02.425465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c5e30-0ed8-4292-a1af-4da4d1d9ca64 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:02.462363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a22a19-2a8e-482a-8aca-48ef79b8faf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:02.463598 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a22a19-2a8e-482a-8aca-48ef79b8faf9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.463932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a22a19-2a8e-482a-8aca-48ef79b8faf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.464216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a22a19-2a8e-482a-8aca-48ef79b8faf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.475050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a22a19-2a8e-482a-8aca-48ef79b8faf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:02.476573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a22a19-2a8e-482a-8aca-48ef79b8faf9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f6a22a19-2a8e-482a-8aca-48ef79b8faf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:00:02.495712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:02.496575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:02.497377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:00:02.497377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:00:02.497377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:00:02.497377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:02.502634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 18/32] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:02.503605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a22a19-2a8e-482a-8aca-48ef79b8faf9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:02.519197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 15/33] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 22 11:00:02 2022] POST /identity/v3/roles => generated 219 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:02.520936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899f841-b683-40a9-ab4c-f12439bc53ef None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:02.604328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:02.618505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a22a19-2a8e-482a-8aca-48ef79b8faf9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:02.655135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899f841-b683-40a9-ab4c-f12439bc53ef None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:02.662022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf38b6-ca17-45f6-b7aa-0dcc2dac287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:02.662994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf38b6-ca17-45f6-b7aa-0dcc2dac287a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.663391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf38b6-ca17-45f6-b7aa-0dcc2dac287a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.663747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf38b6-ca17-45f6-b7aa-0dcc2dac287a None admin] PATH_INFO: `/v3/users/9a8c4f38edad4724ac733753692dda22` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.665550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf38b6-ca17-45f6-b7aa-0dcc2dac287a None admin] RBAC: Authorizing `identity:get_user(user_id=9a8c4f38edad4724ac733753692dda22)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:02.667431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf38b6-ca17-45f6-b7aa-0dcc2dac287a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:02.669154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 19/34] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:02 2022] GET /identity/v3/users/9a8c4f38edad4724ac733753692dda22 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:02.670188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf38b6-ca17-45f6-b7aa-0dcc2dac287a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:02.687728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:02.698597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d2da4-b6ca-404b-b8fe-ee1e126c98b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:02.699827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d2da4-b6ca-404b-b8fe-ee1e126c98b5 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.700522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d2da4-b6ca-404b-b8fe-ee1e126c98b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.700985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d2da4-b6ca-404b-b8fe-ee1e126c98b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.702412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d2da4-b6ca-404b-b8fe-ee1e126c98b5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:02.704787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d2da4-b6ca-404b-b8fe-ee1e126c98b5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:02.720316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 16/35] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 22 11:00:02 2022] POST /identity/v3/roles => generated 225 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:02.721574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d2da4-b6ca-404b-b8fe-ee1e126c98b5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:02.839922 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d2da4-b6ca-404b-b8fe-ee1e126c98b5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:02.872512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf38b6-ca17-45f6-b7aa-0dcc2dac287a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:02.894073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3eb3bf-15a2-45fb-8b5c-c24eb632149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:02.895386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3eb3bf-15a2-45fb-8b5c-c24eb632149d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.895707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3eb3bf-15a2-45fb-8b5c-c24eb632149d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.895996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3eb3bf-15a2-45fb-8b5c-c24eb632149d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.897352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3eb3bf-15a2-45fb-8b5c-c24eb632149d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:02.899480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3eb3bf-15a2-45fb-8b5c-c24eb632149d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:02.901773 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 17/36] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:02.903164 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629283f-051a-499b-9244-d0f5adafc22b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.903496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629283f-051a-499b-9244-d0f5adafc22b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.905284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629283f-051a-499b-9244-d0f5adafc22b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.911435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 18/37] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:02.912765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629283f-051a-499b-9244-d0f5adafc22b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:02.934903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c95dbe-5347-43d5-a2fa-874e90e782b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:02.936090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c95dbe-5347-43d5-a2fa-874e90e782b4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.936606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c95dbe-5347-43d5-a2fa-874e90e782b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.937198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c95dbe-5347-43d5-a2fa-874e90e782b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.938264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c95dbe-5347-43d5-a2fa-874e90e782b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:02.938979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c95dbe-5347-43d5-a2fa-874e90e782b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:02.940716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c95dbe-5347-43d5-a2fa-874e90e782b4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:02.966579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 20/38] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:02.968085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f054-9149-484a-bab4-a0193502a1c6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:02.968905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f054-9149-484a-bab4-a0193502a1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:02.969186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f054-9149-484a-bab4-a0193502a1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:02.990528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0734f054-9149-484a-bab4-a0193502a1c6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:02.991844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0734f054-9149-484a-bab4-a0193502a1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:02.993956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:03.005450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:03.044565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629283f-051a-499b-9244-d0f5adafc22b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:03.073546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 21/39] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:03.074377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f054-9149-484a-bab4-a0193502a1c6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:03.087029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa05757-0620-4130-8440-95271525d2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:03.088260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa05757-0620-4130-8440-95271525d2ad None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:03.088893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa05757-0620-4130-8440-95271525d2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:03.089283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa05757-0620-4130-8440-95271525d2ad None admin] PATH_INFO: `/v3/domains/default/users/9a8c4f38edad4724ac733753692dda22/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:03.091622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa05757-0620-4130-8440-95271525d2ad None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9a8c4f38edad4724ac733753692dda22, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:03.093756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa05757-0620-4130-8440-95271525d2ad None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:03.113762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 19/40] 10.222.0.20 () {58 vars in 1361 bytes} [Fri Jul 22 11:00:02 2022] PUT /identity/v3/domains/default/users/9a8c4f38edad4724ac733753692dda22/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:03.114806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa05757-0620-4130-8440-95271525d2ad None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:03.198211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f054-9149-484a-bab4-a0193502a1c6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:03.246512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a8873-43d0-49b7-8e63-76bc14d261ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:03.247688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a8873-43d0-49b7-8e63-76bc14d261ce None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:03.248570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a8873-43d0-49b7-8e63-76bc14d261ce None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:03.248973 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a8873-43d0-49b7-8e63-76bc14d261ce None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:03.250163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a8873-43d0-49b7-8e63-76bc14d261ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:03.252660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a8873-43d0-49b7-8e63-76bc14d261ce None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:03.255541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa05757-0620-4130-8440-95271525d2ad None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:03.276981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:03.287367 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:03.311603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e60dc6-d6a9-410b-ba12-a828b5f275c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:03.312738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e60dc6-d6a9-410b-ba12-a828b5f275c9 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:03.313302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e60dc6-d6a9-410b-ba12-a828b5f275c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:03.313851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e60dc6-d6a9-410b-ba12-a828b5f275c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:03.314846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e60dc6-d6a9-410b-ba12-a828b5f275c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:03.316648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e60dc6-d6a9-410b-ba12-a828b5f275c9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:03.333376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 20/41] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 22 11:00:03 2022] POST /identity/v3/roles => generated 223 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:03.357920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d87fd-53fc-4435-ac3e-667f15dd65ac None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:03.358233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d87fd-53fc-4435-ac3e-667f15dd65ac None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:03.358502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d87fd-53fc-4435-ac3e-667f15dd65ac None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:03.360349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 21/42] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:00:03.366099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76097-3da3-4b40-8f23-29c66bf2fa01 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:03.366398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76097-3da3-4b40-8f23-29c66bf2fa01 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:03.366688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76097-3da3-4b40-8f23-29c66bf2fa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:03.385075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-41c76097-3da3-4b40-8f23-29c66bf2fa01 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:03.386717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-41c76097-3da3-4b40-8f23-29c66bf2fa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:03.406867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 22/43] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:00:03 2022] POST /identity/v3/projects => generated 301 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:03.481962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 22/44] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:03.506437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a8873-43d0-49b7-8e63-76bc14d261ce None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:03.649105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a8873-43d0-49b7-8e63-76bc14d261ce None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:03.691242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ced58b-e4e2-40d1-bc5d-2c1455f3258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:03.692086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ced58b-e4e2-40d1-bc5d-2c1455f3258d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:03.692378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ced58b-e4e2-40d1-bc5d-2c1455f3258d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:03.692658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ced58b-e4e2-40d1-bc5d-2c1455f3258d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:03.693923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ced58b-e4e2-40d1-bc5d-2c1455f3258d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:03.695537 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ced58b-e4e2-40d1-bc5d-2c1455f3258d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:03.697335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 23/45] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:03.703618 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76097-3da3-4b40-8f23-29c66bf2fa01 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:03.818570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76097-3da3-4b40-8f23-29c66bf2fa01 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:03.821013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:03.961593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac031c-660d-417d-9af4-be3558b39883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:03.962275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac031c-660d-417d-9af4-be3558b39883 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:03.962568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac031c-660d-417d-9af4-be3558b39883 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:03.962858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac031c-660d-417d-9af4-be3558b39883 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:03.963625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac031c-660d-417d-9af4-be3558b39883 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:03.965105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac031c-660d-417d-9af4-be3558b39883 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:03.994795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 23/46] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:00:03 2022] POST /identity/v3/projects => generated 312 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:05.211427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330f60-35d8-42b4-8811-5a598b84ed73 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:05.211949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330f60-35d8-42b4-8811-5a598b84ed73 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:05.212033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330f60-35d8-42b4-8811-5a598b84ed73 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:05.213382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 24/47] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:05.218517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38f5d9-0792-49ef-86da-45530db9c1d0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:05.218926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38f5d9-0792-49ef-86da-45530db9c1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:05.219101 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38f5d9-0792-49ef-86da-45530db9c1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:05.235756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f38f5d9-0792-49ef-86da-45530db9c1d0 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:05.236955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f38f5d9-0792-49ef-86da-45530db9c1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:05.307706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 24/48] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:05.331317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330f60-35d8-42b4-8811-5a598b84ed73 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:05.408327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:05.445241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330f60-35d8-42b4-8811-5a598b84ed73 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:05.478553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749dd74-6365-43e4-8e2f-6623da60d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:05.479306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749dd74-6365-43e4-8e2f-6623da60d06b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:05.479602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749dd74-6365-43e4-8e2f-6623da60d06b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:05.479884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749dd74-6365-43e4-8e2f-6623da60d06b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:05.491764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749dd74-6365-43e4-8e2f-6623da60d06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f749dd74-6365-43e4-8e2f-6623da60d06b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:05.501105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:05.501993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:05.502738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 25/49] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:05.508401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38f5d9-0792-49ef-86da-45530db9c1d0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:05.590142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:05.609149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38f5d9-0792-49ef-86da-45530db9c1d0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:05.643606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb734a0e-73d0-4fa3-ae36-15519b0e8862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:05.644294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb734a0e-73d0-4fa3-ae36-15519b0e8862 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:05.644587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb734a0e-73d0-4fa3-ae36-15519b0e8862 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:05.644869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb734a0e-73d0-4fa3-ae36-15519b0e8862 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:05.645644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb734a0e-73d0-4fa3-ae36-15519b0e8862 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:05.645957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb734a0e-73d0-4fa3-ae36-15519b0e8862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:05.647378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb734a0e-73d0-4fa3-ae36-15519b0e8862 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:05.661194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 25/50] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:05.666893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749dd74-6365-43e4-8e2f-6623da60d06b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:05.870749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749dd74-6365-43e4-8e2f-6623da60d06b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:05.895221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:05.904597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02bd98-8fad-4302-b10e-057b0963b359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:05.905972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02bd98-8fad-4302-b10e-057b0963b359 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:05.906264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02bd98-8fad-4302-b10e-057b0963b359 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:05.906544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02bd98-8fad-4302-b10e-057b0963b359 None admin] PATH_INFO: `/v3/users/9a8c4f38edad4724ac733753692dda22` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:05.907866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02bd98-8fad-4302-b10e-057b0963b359 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8c4f38edad4724ac733753692dda22)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:05.909332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02bd98-8fad-4302-b10e-057b0963b359 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:05.910973 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 26/51] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:05 2022] GET /identity/v3/users/9a8c4f38edad4724ac733753692dda22 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:05.916405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb734a0e-73d0-4fa3-ae36-15519b0e8862 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:06.021700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb734a0e-73d0-4fa3-ae36-15519b0e8862 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:06.056877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b097104-4f8f-45f6-a011-1f0e470a6589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:06.057673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b097104-4f8f-45f6-a011-1f0e470a6589 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:06.057963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b097104-4f8f-45f6-a011-1f0e470a6589 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:06.058242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b097104-4f8f-45f6-a011-1f0e470a6589 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:06.059450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b097104-4f8f-45f6-a011-1f0e470a6589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:06.060905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b097104-4f8f-45f6-a011-1f0e470a6589 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:06.062550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 26/52] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:06.068268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02bd98-8fad-4302-b10e-057b0963b359 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:06.096417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fe392-4acb-4da5-9804-9d2d34821108 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:06.096728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fe392-4acb-4da5-9804-9d2d34821108 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:06.097019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fe392-4acb-4da5-9804-9d2d34821108 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:06.098411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 27/53] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:00:06.103378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0fa5e-7142-4a9d-a4c7-b3c60cbcd683 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:06.103678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0fa5e-7142-4a9d-a4c7-b3c60cbcd683 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:06.103962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0fa5e-7142-4a9d-a4c7-b3c60cbcd683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:06.120738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9cf0fa5e-7142-4a9d-a4c7-b3c60cbcd683 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:06.122404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9cf0fa5e-7142-4a9d-a4c7-b3c60cbcd683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:06.136062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:06.180464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02bd98-8fad-4302-b10e-057b0963b359 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:06.197733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 28/54] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:06.213559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1096642-30f3-4ad5-9009-19ac89213c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:06.214230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1096642-30f3-4ad5-9009-19ac89213c7b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:06.214521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1096642-30f3-4ad5-9009-19ac89213c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:06.214803 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1096642-30f3-4ad5-9009-19ac89213c7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:06.215679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1096642-30f3-4ad5-9009-19ac89213c7b None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.domain.id=default, user.id=9a8c4f38edad4724ac733753692dda22) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:06.215997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1096642-30f3-4ad5-9009-19ac89213c7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:06.217469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1096642-30f3-4ad5-9009-19ac89213c7b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:06.221920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0fa5e-7142-4a9d-a4c7-b3c60cbcd683 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:06.226853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 27/55] 10.222.0.20 () {56 vars in 1322 bytes} [Fri Jul 22 11:00:06 2022] GET /identity/v3/role_assignments?user.id=9a8c4f38edad4724ac733753692dda22&scope.domain.id=default&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:06.332043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0fa5e-7142-4a9d-a4c7-b3c60cbcd683 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:06.375935 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3baa37c-52f1-44f5-a206-0da0d75e0bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:06.377062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3baa37c-52f1-44f5-a206-0da0d75e0bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:06.377498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3baa37c-52f1-44f5-a206-0da0d75e0bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:06.377929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3baa37c-52f1-44f5-a206-0da0d75e0bf1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:06.379772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3baa37c-52f1-44f5-a206-0da0d75e0bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:06.382129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3baa37c-52f1-44f5-a206-0da0d75e0bf1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:06.385174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 29/56] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:06.395320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1096642-30f3-4ad5-9009-19ac89213c7b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:06.453191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:06.495500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1096642-30f3-4ad5-9009-19ac89213c7b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:06.529149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4da3c1-6fff-4450-847e-84c18ddf298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:06.529813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4da3c1-6fff-4450-847e-84c18ddf298a None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:06.530103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4da3c1-6fff-4450-847e-84c18ddf298a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:06.530409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4da3c1-6fff-4450-847e-84c18ddf298a None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:06.531131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4da3c1-6fff-4450-847e-84c18ddf298a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:06.532628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4da3c1-6fff-4450-847e-84c18ddf298a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:06.557109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 28/57] 10.222.0.20 () {60 vars in 1140 bytes} [Fri Jul 22 11:00:06 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:08.610130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ff0e1-1e0b-41f2-847f-b8139a2ea278 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:08.610632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ff0e1-1e0b-41f2-847f-b8139a2ea278 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:08.610732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ff0e1-1e0b-41f2-847f-b8139a2ea278 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:08.612105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 30/58] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:08.617416 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c32ed-7b67-4652-9038-211876321046 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:08.617849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c32ed-7b67-4652-9038-211876321046 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:08.617991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c32ed-7b67-4652-9038-211876321046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:08.636476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ef5c32ed-7b67-4652-9038-211876321046 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:08.638296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ef5c32ed-7b67-4652-9038-211876321046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:08.706119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 29/59] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:08.731660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ff0e1-1e0b-41f2-847f-b8139a2ea278 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:08.748858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:08.862916 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ff0e1-1e0b-41f2-847f-b8139a2ea278 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:08.907414 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900c52b-ee0d-407e-8033-b3e4ffaba344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:08.908583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900c52b-ee0d-407e-8033-b3e4ffaba344 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:08.909008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900c52b-ee0d-407e-8033-b3e4ffaba344 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:08.909428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900c52b-ee0d-407e-8033-b3e4ffaba344 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:08.911109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900c52b-ee0d-407e-8033-b3e4ffaba344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:08.913522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900c52b-ee0d-407e-8033-b3e4ffaba344 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:08.916225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 31/60] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:08.922432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c32ed-7b67-4652-9038-211876321046 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:08.966587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:09.028817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c32ed-7b67-4652-9038-211876321046 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:09.064192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f1d0e-77eb-4f63-b15c-801cbfc499d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:09.065073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f1d0e-77eb-4f63-b15c-801cbfc499d1 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:09.065386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f1d0e-77eb-4f63-b15c-801cbfc499d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:09.065669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f1d0e-77eb-4f63-b15c-801cbfc499d1 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:09.066394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f1d0e-77eb-4f63-b15c-801cbfc499d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:09.067908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f1d0e-77eb-4f63-b15c-801cbfc499d1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:09.094291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 30/61] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:00:08 2022] POST /identity/v3/users => generated 275 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:12.449105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e61d95-a633-4266-b43b-5144d342bb52 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:12.450123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e61d95-a633-4266-b43b-5144d342bb52 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:12.450614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e61d95-a633-4266-b43b-5144d342bb52 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:12.452578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 32/62] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:12.457646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b02c3-45a3-498f-a0d8-b7dbcc7ecbd6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:12.458066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b02c3-45a3-498f-a0d8-b7dbcc7ecbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:12.458222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b02c3-45a3-498f-a0d8-b7dbcc7ecbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:12.481714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-052b02c3-45a3-498f-a0d8-b7dbcc7ecbd6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:12.483195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-052b02c3-45a3-498f-a0d8-b7dbcc7ecbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:12.593959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 31/63] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:12.630652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e61d95-a633-4266-b43b-5144d342bb52 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:12.654162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec45b6-24da-4eed-901a-008619ef6585 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:12.655667 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec45b6-24da-4eed-901a-008619ef6585 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:12.656235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec45b6-24da-4eed-901a-008619ef6585 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:12.658707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 32/64] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:12.664380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642336-2c6a-4317-8ca7-6deffdebf77e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:12.664675 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642336-2c6a-4317-8ca7-6deffdebf77e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:12.664961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642336-2c6a-4317-8ca7-6deffdebf77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:12.689752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-49642336-2c6a-4317-8ca7-6deffdebf77e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:12.691000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-49642336-2c6a-4317-8ca7-6deffdebf77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:12.731763 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:12.773200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:12.792851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 33/65] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:12.794693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880eaba-e658-453d-a672-4c0dcf65c142 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:12.795206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880eaba-e658-453d-a672-4c0dcf65c142 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:12.796302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880eaba-e658-453d-a672-4c0dcf65c142 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:12.801992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 34/66] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:12.803003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e61d95-a633-4266-b43b-5144d342bb52 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:12.804530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e357c-aacb-4585-bd7a-2d5a3aac59fe None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:12.805310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e357c-aacb-4585-bd7a-2d5a3aac59fe None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:12.806118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e357c-aacb-4585-bd7a-2d5a3aac59fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:12.837012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3a6e357c-aacb-4585-bd7a-2d5a3aac59fe None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:12.838226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3a6e357c-aacb-4585-bd7a-2d5a3aac59fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:12.867624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3030c3-9506-4ce2-8572-b7d296bd56b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:12.868793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3030c3-9506-4ce2-8572-b7d296bd56b7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:12.869783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3030c3-9506-4ce2-8572-b7d296bd56b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:12.874291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3030c3-9506-4ce2-8572-b7d296bd56b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:12.883305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3030c3-9506-4ce2-8572-b7d296bd56b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2f3030c3-9506-4ce2-8572-b7d296bd56b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:12.897796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:12.898719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:12.899527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 33/67] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:12.900710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3030c3-9506-4ce2-8572-b7d296bd56b7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:12.932077 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 35/68] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:12.934142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a4ba9-ae49-4edb-98e5-b17a93d4ecdc None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:12.934633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a4ba9-ae49-4edb-98e5-b17a93d4ecdc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:12.935111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a4ba9-ae49-4edb-98e5-b17a93d4ecdc None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:12.939397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 36/69] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:12.939397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a4ba9-ae49-4edb-98e5-b17a93d4ecdc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.022855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3030c3-9506-4ce2-8572-b7d296bd56b7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.049283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:13.057566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748f88c-ee07-4b4a-8d19-c6c552b4f088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.058368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748f88c-ee07-4b4a-8d19-c6c552b4f088 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.058681 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748f88c-ee07-4b4a-8d19-c6c552b4f088 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.058960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748f88c-ee07-4b4a-8d19-c6c552b4f088 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.065463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a4ba9-ae49-4edb-98e5-b17a93d4ecdc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.069280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748f88c-ee07-4b4a-8d19-c6c552b4f088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4748f88c-ee07-4b4a-8d19-c6c552b4f088 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:13.077552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:13.078644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:13.079463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 34/70] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:13.080138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590db71b-6cc9-4c46-8b1e-cd5114e249b2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.080427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590db71b-6cc9-4c46-8b1e-cd5114e249b2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.080708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590db71b-6cc9-4c46-8b1e-cd5114e249b2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.081982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 35/71] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:13.083019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31169584-c2af-4152-9d51-cd54164ce3f2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.083480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31169584-c2af-4152-9d51-cd54164ce3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.083750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31169584-c2af-4152-9d51-cd54164ce3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.089636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:13.102876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-31169584-c2af-4152-9d51-cd54164ce3f2 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:13.104136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-31169584-c2af-4152-9d51-cd54164ce3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:13.108184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe60e42-0daa-4597-acd5-924a4d931e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.108864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe60e42-0daa-4597-acd5-924a4d931e5c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.109155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe60e42-0daa-4597-acd5-924a4d931e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.109442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe60e42-0daa-4597-acd5-924a4d931e5c None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.110210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe60e42-0daa-4597-acd5-924a4d931e5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:13.110520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe60e42-0daa-4597-acd5-924a4d931e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:13.111957 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe60e42-0daa-4597-acd5-924a4d931e5c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:13.123886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 37/72] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:13.124717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe60e42-0daa-4597-acd5-924a4d931e5c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.170580 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 36/73] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:13.171835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31169584-c2af-4152-9d51-cd54164ce3f2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.230017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe60e42-0daa-4597-acd5-924a4d931e5c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.263787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd722716-cd15-493b-9f76-9daea63274ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.265248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd722716-cd15-493b-9f76-9daea63274ff None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.265542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd722716-cd15-493b-9f76-9daea63274ff None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.265820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd722716-cd15-493b-9f76-9daea63274ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.270883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31169584-c2af-4152-9d51-cd54164ce3f2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.277104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd722716-cd15-493b-9f76-9daea63274ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fd722716-cd15-493b-9f76-9daea63274ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:13.286536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:13.287506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:13.288267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 38/74] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:13.289112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da48270-ea4d-4d40-8247-0379a9a344f8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.291591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da48270-ea4d-4d40-8247-0379a9a344f8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.291591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da48270-ea4d-4d40-8247-0379a9a344f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.301195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:13.306654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ca867-d2d5-4e2e-b8a5-13815adb6c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.306963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8da48270-ea4d-4d40-8247-0379a9a344f8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:13.307373 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ca867-d2d5-4e2e-b8a5-13815adb6c2a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.307663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ca867-d2d5-4e2e-b8a5-13815adb6c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.307942 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ca867-d2d5-4e2e-b8a5-13815adb6c2a None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.308243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8da48270-ea4d-4d40-8247-0379a9a344f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:13.308729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ca867-d2d5-4e2e-b8a5-13815adb6c2a None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:13.309171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ca867-d2d5-4e2e-b8a5-13815adb6c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:13.310286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:13.310581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ca867-d2d5-4e2e-b8a5-13815adb6c2a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:13.319830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 37/75] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:13.320678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ca867-d2d5-4e2e-b8a5-13815adb6c2a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.385293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 39/76] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:13.386171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da48270-ea4d-4d40-8247-0379a9a344f8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.445819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ca867-d2d5-4e2e-b8a5-13815adb6c2a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.480212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2efca-2c03-4be0-a52b-b749ab50f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.481087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2efca-2c03-4be0-a52b-b749ab50f709 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.481382 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2efca-2c03-4be0-a52b-b749ab50f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.481669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2efca-2c03-4be0-a52b-b749ab50f709 None admin] PATH_INFO: `/v3/users/9a8c4f38edad4724ac733753692dda22` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.482910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2efca-2c03-4be0-a52b-b749ab50f709 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8c4f38edad4724ac733753692dda22)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:13.484465 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2efca-2c03-4be0-a52b-b749ab50f709 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:13.486128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 38/77] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/users/9a8c4f38edad4724ac733753692dda22 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:13.486912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2efca-2c03-4be0-a52b-b749ab50f709 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.495547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da48270-ea4d-4d40-8247-0379a9a344f8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.531231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99784477-e77f-4dcf-b601-9b73929ccfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.531966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99784477-e77f-4dcf-b601-9b73929ccfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.532257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99784477-e77f-4dcf-b601-9b73929ccfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.532540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99784477-e77f-4dcf-b601-9b73929ccfc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.539786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:13.544810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99784477-e77f-4dcf-b601-9b73929ccfc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-99784477-e77f-4dcf-b601-9b73929ccfc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:13.553587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:13.554566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:13.555453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:13.556266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:13.556352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 40/78] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:13.556352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99784477-e77f-4dcf-b601-9b73929ccfc4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.599103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2efca-2c03-4be0-a52b-b749ab50f709 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.634928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c8abf-31e3-4ad9-ba99-35dc4eee7234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.635640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c8abf-31e3-4ad9-ba99-35dc4eee7234 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.635934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c8abf-31e3-4ad9-ba99-35dc4eee7234 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.636217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c8abf-31e3-4ad9-ba99-35dc4eee7234 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.636977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c8abf-31e3-4ad9-ba99-35dc4eee7234 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:13.637286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c8abf-31e3-4ad9-ba99-35dc4eee7234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:13.638686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c8abf-31e3-4ad9-ba99-35dc4eee7234 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:13.646701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 39/79] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:13.647544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c8abf-31e3-4ad9-ba99-35dc4eee7234 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.671835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99784477-e77f-4dcf-b601-9b73929ccfc4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.706311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c841e0-64c7-4cc1-bd73-d1be832687e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.707193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c841e0-64c7-4cc1-bd73-d1be832687e5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.707513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c841e0-64c7-4cc1-bd73-d1be832687e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.707791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c841e0-64c7-4cc1-bd73-d1be832687e5 None admin] PATH_INFO: `/v3/users/6e889b4f5d384fef831673b867e27bfa` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.724356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c841e0-64c7-4cc1-bd73-d1be832687e5 None admin] RBAC: Authorizing `identity:get_user(user_id=6e889b4f5d384fef831673b867e27bfa)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:13.729386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c841e0-64c7-4cc1-bd73-d1be832687e5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:13.731422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 41/80] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/users/6e889b4f5d384fef831673b867e27bfa => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:13.732022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c841e0-64c7-4cc1-bd73-d1be832687e5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.752793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c8abf-31e3-4ad9-ba99-35dc4eee7234 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.778693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:13.789976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c0655-55bd-4ed8-a93a-866e594bd36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.790833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c0655-55bd-4ed8-a93a-866e594bd36a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.791126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c0655-55bd-4ed8-a93a-866e594bd36a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.791440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c0655-55bd-4ed8-a93a-866e594bd36a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.792644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c0655-55bd-4ed8-a93a-866e594bd36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:13.794097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c0655-55bd-4ed8-a93a-866e594bd36a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:13.794568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:13.802711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 40/81] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:13.802711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c0655-55bd-4ed8-a93a-866e594bd36a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:13.839853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c841e0-64c7-4cc1-bd73-d1be832687e5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.874576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720c8d9-4a09-4ee5-8165-312afd1bdf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.875353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720c8d9-4a09-4ee5-8165-312afd1bdf27 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.875656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720c8d9-4a09-4ee5-8165-312afd1bdf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.875944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720c8d9-4a09-4ee5-8165-312afd1bdf27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.900809 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c0655-55bd-4ed8-a93a-866e594bd36a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:13.935383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0384a45-cb31-45dc-9b73-11d61c21911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:13.936071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0384a45-cb31-45dc-9b73-11d61c21911c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:13.937017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0384a45-cb31-45dc-9b73-11d61c21911c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:13.937310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0384a45-cb31-45dc-9b73-11d61c21911c None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:13.938084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0384a45-cb31-45dc-9b73-11d61c21911c None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:13.938480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0384a45-cb31-45dc-9b73-11d61c21911c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:13.939930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0384a45-cb31-45dc-9b73-11d61c21911c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:13.947036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720c8d9-4a09-4ee5-8165-312afd1bdf27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:13.948574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720c8d9-4a09-4ee5-8165-312afd1bdf27 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:13.948746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 41/82] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:13.949586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0384a45-cb31-45dc-9b73-11d61c21911c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:14.004654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:14.017537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:14.054176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 42/83] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:14.054176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720c8d9-4a09-4ee5-8165-312afd1bdf27 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:14.067767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0384a45-cb31-45dc-9b73-11d61c21911c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:14.105451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e257b7-862f-4947-b826-beda0b283bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:14.106804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e257b7-862f-4947-b826-beda0b283bdc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:14.107353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e257b7-862f-4947-b826-beda0b283bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:14.107807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e257b7-862f-4947-b826-beda0b283bdc None admin] PATH_INFO: `/v3/users/6e889b4f5d384fef831673b867e27bfa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:14.109743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e257b7-862f-4947-b826-beda0b283bdc None admin] RBAC: Authorizing `identity:get_user(user_id=6e889b4f5d384fef831673b867e27bfa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:14.111203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e257b7-862f-4947-b826-beda0b283bdc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:14.112953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 42/84] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:13 2022] GET /identity/v3/users/6e889b4f5d384fef831673b867e27bfa => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:14.113755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e257b7-862f-4947-b826-beda0b283bdc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:14.156316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720c8d9-4a09-4ee5-8165-312afd1bdf27 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:14.191127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765a537-8393-44ac-af6a-ccf3d93d63bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:14.191818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765a537-8393-44ac-af6a-ccf3d93d63bf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:14.192114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765a537-8393-44ac-af6a-ccf3d93d63bf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:14.192398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765a537-8393-44ac-af6a-ccf3d93d63bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:14.217857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:14.230989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e257b7-862f-4947-b826-beda0b283bdc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:14.255646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:14.265867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765a537-8393-44ac-af6a-ccf3d93d63bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:14.267189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765a537-8393-44ac-af6a-ccf3d93d63bf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:14.267878 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e43cc-7c7f-4aab-8ec3-f7bc9bf57801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:14.268561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e43cc-7c7f-4aab-8ec3-f7bc9bf57801 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:14.268859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e43cc-7c7f-4aab-8ec3-f7bc9bf57801 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:14.269146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e43cc-7c7f-4aab-8ec3-f7bc9bf57801 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:14.269882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e43cc-7c7f-4aab-8ec3-f7bc9bf57801 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:14.271384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e43cc-7c7f-4aab-8ec3-f7bc9bf57801 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:14.298614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 43/85] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:00:14 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:14.299493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e43cc-7c7f-4aab-8ec3-f7bc9bf57801 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:14.390184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 43/86] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:14.390184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765a537-8393-44ac-af6a-ccf3d93d63bf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:14.418836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e43cc-7c7f-4aab-8ec3-f7bc9bf57801 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:14.450773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:14.459117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d9816-639b-4a88-9fc7-0b7fc6afbdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:14.459451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d9816-639b-4a88-9fc7-0b7fc6afbdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:14.459747 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d9816-639b-4a88-9fc7-0b7fc6afbdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:14.460032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d9816-639b-4a88-9fc7-0b7fc6afbdd5 None admin] PATH_INFO: `/v3/users/6e889b4f5d384fef831673b867e27bfa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:14.461896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d9816-639b-4a88-9fc7-0b7fc6afbdd5 None admin] RBAC: Authorizing `identity:get_user(user_id=6e889b4f5d384fef831673b867e27bfa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:14.463118 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d9816-639b-4a88-9fc7-0b7fc6afbdd5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:14.464842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 44/87] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:14 2022] GET /identity/v3/users/6e889b4f5d384fef831673b867e27bfa => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:14.465647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d9816-639b-4a88-9fc7-0b7fc6afbdd5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:14.516394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765a537-8393-44ac-af6a-ccf3d93d63bf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:14.535891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:14.558078 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9907c-7fee-48ed-b965-abaeaaa539b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:14.558993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9907c-7fee-48ed-b965-abaeaaa539b4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:14.559445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9907c-7fee-48ed-b965-abaeaaa539b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:14.559884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9907c-7fee-48ed-b965-abaeaaa539b4 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:14.561200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9907c-7fee-48ed-b965-abaeaaa539b4 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:14.562887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9907c-7fee-48ed-b965-abaeaaa539b4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:14.565502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 44/88] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:14 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:14.566093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9907c-7fee-48ed-b965-abaeaaa539b4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:14.612971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d9816-639b-4a88-9fc7-0b7fc6afbdd5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:14.656579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172756f9-5056-4758-95f6-80d096cb60c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:14.657170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172756f9-5056-4758-95f6-80d096cb60c6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:14.657463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172756f9-5056-4758-95f6-80d096cb60c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:14.657768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172756f9-5056-4758-95f6-80d096cb60c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:14.707108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9907c-7fee-48ed-b965-abaeaaa539b4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:14.741450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:14.753741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97fc14-91a4-45f2-9f20-b710486df682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:14.754857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97fc14-91a4-45f2-9f20-b710486df682 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:14.755149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97fc14-91a4-45f2-9f20-b710486df682 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:14.755470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97fc14-91a4-45f2-9f20-b710486df682 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:14.756977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97fc14-91a4-45f2-9f20-b710486df682 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:14.758450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97fc14-91a4-45f2-9f20-b710486df682 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:14.760421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 45/89] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:14 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:14.761202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97fc14-91a4-45f2-9f20-b710486df682 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:14.769666 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172756f9-5056-4758-95f6-80d096cb60c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:14.773517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172756f9-5056-4758-95f6-80d096cb60c6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:14.853190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:14.883389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97fc14-91a4-45f2-9f20-b710486df682 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:14.916208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 45/90] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:14.917096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172756f9-5056-4758-95f6-80d096cb60c6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:14.931475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a85de-f950-4fdf-8b13-48e3bde0f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:14.932434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a85de-f950-4fdf-8b13-48e3bde0f299 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:14.933148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a85de-f950-4fdf-8b13-48e3bde0f299 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:14.933628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a85de-f950-4fdf-8b13-48e3bde0f299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:15.017175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:15.027398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a85de-f950-4fdf-8b13-48e3bde0f299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:15.029053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a85de-f950-4fdf-8b13-48e3bde0f299 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:15.034528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172756f9-5056-4758-95f6-80d096cb60c6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:15.072882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6210d-73ad-447e-8638-3e2db26d3e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:15.073904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6210d-73ad-447e-8638-3e2db26d3e0e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:15.074412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6210d-73ad-447e-8638-3e2db26d3e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:15.074901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6210d-73ad-447e-8638-3e2db26d3e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:15.075990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6210d-73ad-447e-8638-3e2db26d3e0e None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701, user.id=9a8c4f38edad4724ac733753692dda22) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:15.076509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6210d-73ad-447e-8638-3e2db26d3e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:15.078184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6210d-73ad-447e-8638-3e2db26d3e0e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:15.084301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 46/91] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:14 2022] GET /identity/v3/role_assignments?user.id=9a8c4f38edad4724ac733753692dda22&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:15.085421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6210d-73ad-447e-8638-3e2db26d3e0e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:15.101554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:15.231586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6210d-73ad-447e-8638-3e2db26d3e0e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:15.241215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 46/92] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:15.245123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a85de-f950-4fdf-8b13-48e3bde0f299 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:15.310253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a22947-8233-4373-a627-1a567af6c092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:15.312449 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a22947-8233-4373-a627-1a567af6c092 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:15.312749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a22947-8233-4373-a627-1a567af6c092 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:15.313037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a22947-8233-4373-a627-1a567af6c092 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:15.313903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a22947-8233-4373-a627-1a567af6c092 None admin] RBAC: Adding query filter params (role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701, user.id=6e889b4f5d384fef831673b867e27bfa) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:15.314223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a22947-8233-4373-a627-1a567af6c092 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:15.315748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a22947-8233-4373-a627-1a567af6c092 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:15.322963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 47/93] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:15 2022] GET /identity/v3/role_assignments?user.id=6e889b4f5d384fef831673b867e27bfa&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:15.323975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a22947-8233-4373-a627-1a567af6c092 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:15.384552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:15.395491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a85de-f950-4fdf-8b13-48e3bde0f299 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:15.438267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb67b37-f2cb-4c84-bd87-23925d30d5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:15.439963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb67b37-f2cb-4c84-bd87-23925d30d5fd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:15.440503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb67b37-f2cb-4c84-bd87-23925d30d5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:15.441216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb67b37-f2cb-4c84-bd87-23925d30d5fd None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:15.443051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb67b37-f2cb-4c84-bd87-23925d30d5fd None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:15.445801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb67b37-f2cb-4c84-bd87-23925d30d5fd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:15.448170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 47/94] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:15 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:15.455160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb67b37-f2cb-4c84-bd87-23925d30d5fd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:15.500199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:15.553051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a22947-8233-4373-a627-1a567af6c092 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:15.615680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb67b37-f2cb-4c84-bd87-23925d30d5fd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:15.628100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ac202-2ddf-457c-bab1-68997c39b6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:15.628999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ac202-2ddf-457c-bab1-68997c39b6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:15.629656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ac202-2ddf-457c-bab1-68997c39b6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:15.629656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ac202-2ddf-457c-bab1-68997c39b6c7 None admin] PATH_INFO: `/v3/projects/000e1f52e4464a07aca1f72426e11112` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:15.630964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ac202-2ddf-457c-bab1-68997c39b6c7 None admin] RBAC: Authorizing `identity:get_project(project_id=000e1f52e4464a07aca1f72426e11112)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:15.632522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ac202-2ddf-457c-bab1-68997c39b6c7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:15.634274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 48/95] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:15 2022] GET /identity/v3/projects/000e1f52e4464a07aca1f72426e11112 => generated 312 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:15.640312 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ac202-2ddf-457c-bab1-68997c39b6c7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:15.664978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906bf10-4909-4e4c-bf61-33c1136b4b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:15.666039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906bf10-4909-4e4c-bf61-33c1136b4b74 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:15.666546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906bf10-4909-4e4c-bf61-33c1136b4b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:15.667023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906bf10-4909-4e4c-bf61-33c1136b4b74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:15.668094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906bf10-4909-4e4c-bf61-33c1136b4b74 None admin] RBAC: Adding query filter params (role.id=2e875b35d6744573ae0287b954108209, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701, user.id=6e889b4f5d384fef831673b867e27bfa) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:15.668611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906bf10-4909-4e4c-bf61-33c1136b4b74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:15.670311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906bf10-4909-4e4c-bf61-33c1136b4b74 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:15.683165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 48/96] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:15 2022] GET /identity/v3/role_assignments?user.id=6e889b4f5d384fef831673b867e27bfa&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=2e875b35d6744573ae0287b954108209 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:15.813569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ac202-2ddf-457c-bab1-68997c39b6c7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:15.859616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b12c27-27f8-4061-b7f6-cd72e01e2667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:15.860300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b12c27-27f8-4061-b7f6-cd72e01e2667 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:15.860597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b12c27-27f8-4061-b7f6-cd72e01e2667 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:15.861005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b12c27-27f8-4061-b7f6-cd72e01e2667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:15.861903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b12c27-27f8-4061-b7f6-cd72e01e2667 None admin] RBAC: Adding query filter params (role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=000e1f52e4464a07aca1f72426e11112, user.id=6e889b4f5d384fef831673b867e27bfa) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:15.862244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b12c27-27f8-4061-b7f6-cd72e01e2667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:15.863891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b12c27-27f8-4061-b7f6-cd72e01e2667 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:15.871849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 49/97] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:15 2022] GET /identity/v3/role_assignments?user.id=6e889b4f5d384fef831673b867e27bfa&scope.project.id=000e1f52e4464a07aca1f72426e11112&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:17.324949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c6bca-0879-492b-b7ca-d338b844a45d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:17.325565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c6bca-0879-492b-b7ca-d338b844a45d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:17.325565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c6bca-0879-492b-b7ca-d338b844a45d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:17.327766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 49/98] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:17.332769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57635c-bade-40a6-8dc1-c76f2f89fc67 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:17.333199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57635c-bade-40a6-8dc1-c76f2f89fc67 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:17.333348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57635c-bade-40a6-8dc1-c76f2f89fc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:17.349261 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:17.360272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6d57635c-bade-40a6-8dc1-c76f2f89fc67 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:17.361505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6d57635c-bade-40a6-8dc1-c76f2f89fc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:17.424723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fd554-5a3a-4bff-a1a1-63d78dc7e087 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:17.425016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fd554-5a3a-4bff-a1a1-63d78dc7e087 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:17.425305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fd554-5a3a-4bff-a1a1-63d78dc7e087 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:17.426959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 50/99] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:17.437027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0787d-d7d5-437f-a5c0-ab08db0a387a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:17.437665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0787d-d7d5-437f-a5c0-ab08db0a387a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:17.437954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0787d-d7d5-437f-a5c0-ab08db0a387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:17.462770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6ad0787d-d7d5-437f-a5c0-ab08db0a387a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:17.464014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6ad0787d-d7d5-437f-a5c0-ab08db0a387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:17.467246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 50/100] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:17.485008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:17.499537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57635c-bade-40a6-8dc1-c76f2f89fc67 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:17.555862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 51/101] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:17.579600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0787d-d7d5-437f-a5c0-ab08db0a387a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:17.689252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0787d-d7d5-437f-a5c0-ab08db0a387a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:17.742781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa524bf0-cde2-42a7-840d-18be6d8ef029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:17.743892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa524bf0-cde2-42a7-840d-18be6d8ef029 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:17.744346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa524bf0-cde2-42a7-840d-18be6d8ef029 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:17.744788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa524bf0-cde2-42a7-840d-18be6d8ef029 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:17.746095 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57635c-bade-40a6-8dc1-c76f2f89fc67 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:17.746314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa524bf0-cde2-42a7-840d-18be6d8ef029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:17.747955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa524bf0-cde2-42a7-840d-18be6d8ef029 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:17.750611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 52/102] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:17.763190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa524bf0-cde2-42a7-840d-18be6d8ef029 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:17.804459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:17.834621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d127a87-abea-4201-9f57-ef9b7630fc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:17.835591 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d127a87-abea-4201-9f57-ef9b7630fc63 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:17.835766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d127a87-abea-4201-9f57-ef9b7630fc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:17.836028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d127a87-abea-4201-9f57-ef9b7630fc63 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:17.844482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:17.866854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d127a87-abea-4201-9f57-ef9b7630fc63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6d127a87-abea-4201-9f57-ef9b7630fc63 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:17.883678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:17.884636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:17.885689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 51/103] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:17.893053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d127a87-abea-4201-9f57-ef9b7630fc63 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:17.895240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa524bf0-cde2-42a7-840d-18be6d8ef029 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:17.932434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5a83b-0aec-4928-997d-2f624dc29fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:17.933381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5a83b-0aec-4928-997d-2f624dc29fe5 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:17.933818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5a83b-0aec-4928-997d-2f624dc29fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:17.934268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5a83b-0aec-4928-997d-2f624dc29fe5 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:17.935167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5a83b-0aec-4928-997d-2f624dc29fe5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:17.937334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5a83b-0aec-4928-997d-2f624dc29fe5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:17.968605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 53/104] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:00:17 2022] POST /identity/v3/projects => generated 302 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:00:18.017843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92bdb0-de8d-442e-b78f-c459b7672c5f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.018419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92bdb0-de8d-442e-b78f-c459b7672c5f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.018866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92bdb0-de8d-442e-b78f-c459b7672c5f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.020647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 54/105] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:00:18.037323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a09d4-ac96-4554-bca4-df3552965aa5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.037795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a09d4-ac96-4554-bca4-df3552965aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.038252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a09d4-ac96-4554-bca4-df3552965aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.066821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0d1a09d4-ac96-4554-bca4-df3552965aa5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:18.068134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0d1a09d4-ac96-4554-bca4-df3552965aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:18.074031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d127a87-abea-4201-9f57-ef9b7630fc63 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:18.141146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922b0fd-922e-4150-94e4-017ea844da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:18.142873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922b0fd-922e-4150-94e4-017ea844da3a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.143748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922b0fd-922e-4150-94e4-017ea844da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.144262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922b0fd-922e-4150-94e4-017ea844da3a None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.145587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922b0fd-922e-4150-94e4-017ea844da3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:18.146712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922b0fd-922e-4150-94e4-017ea844da3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:18.148872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922b0fd-922e-4150-94e4-017ea844da3a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:18.161845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 52/106] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:18.164639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b258d12-ec33-4170-be61-358dd28d720e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.165413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b258d12-ec33-4170-be61-358dd28d720e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.166663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b258d12-ec33-4170-be61-358dd28d720e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.168692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 53/107] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:18.169955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b258d12-ec33-4170-be61-358dd28d720e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:18.175365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 55/108] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:18.180861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e0ebc-f007-47bd-8f07-975216095c6b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.181222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e0ebc-f007-47bd-8f07-975216095c6b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.181561 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e0ebc-f007-47bd-8f07-975216095c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.194210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:18.207859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6a9e0ebc-f007-47bd-8f07-975216095c6b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:18.209106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6a9e0ebc-f007-47bd-8f07-975216095c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:18.258092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:18.310739 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 56/109] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:18.313458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e0ebc-f007-47bd-8f07-975216095c6b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:18.315771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b258d12-ec33-4170-be61-358dd28d720e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:18.372457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42ffc8-5b11-460a-97df-3eb25518cb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:18.374750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42ffc8-5b11-460a-97df-3eb25518cb00 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.375389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42ffc8-5b11-460a-97df-3eb25518cb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.375872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42ffc8-5b11-460a-97df-3eb25518cb00 None admin] PATH_INFO: `/v3/users/06f60bd270b04444b3c9d9fe277097bc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.391369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42ffc8-5b11-460a-97df-3eb25518cb00 None admin] RBAC: Authorizing `identity:get_user(user_id=06f60bd270b04444b3c9d9fe277097bc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:18.393192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42ffc8-5b11-460a-97df-3eb25518cb00 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:18.395207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 54/110] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:18 2022] GET /identity/v3/users/06f60bd270b04444b3c9d9fe277097bc => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:18.396290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42ffc8-5b11-460a-97df-3eb25518cb00 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:18.502034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42ffc8-5b11-460a-97df-3eb25518cb00 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:18.519868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:18.522396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e0ebc-f007-47bd-8f07-975216095c6b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:18.551123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a322f7f-5209-4b08-8d98-ad44e5c37687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:18.552379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a322f7f-5209-4b08-8d98-ad44e5c37687 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.553291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a322f7f-5209-4b08-8d98-ad44e5c37687 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.554570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a322f7f-5209-4b08-8d98-ad44e5c37687 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.574730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a322f7f-5209-4b08-8d98-ad44e5c37687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4a322f7f-5209-4b08-8d98-ad44e5c37687 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:18.587294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:18.588226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:18.588996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:18.606743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 55/111] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:18.608793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a322f7f-5209-4b08-8d98-ad44e5c37687 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:18.610237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01aa3f-c3c8-4c39-853a-13c344bcbda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:18.611321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01aa3f-c3c8-4c39-853a-13c344bcbda7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.611820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01aa3f-c3c8-4c39-853a-13c344bcbda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.612308 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01aa3f-c3c8-4c39-853a-13c344bcbda7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.620906 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:18.633983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01aa3f-c3c8-4c39-853a-13c344bcbda7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8e01aa3f-c3c8-4c39-853a-13c344bcbda7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:18.650182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:18.651098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:18.652314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 57/112] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:18.659871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01aa3f-c3c8-4c39-853a-13c344bcbda7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:18.767951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a322f7f-5209-4b08-8d98-ad44e5c37687 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:18.820609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab74c4-9be9-441e-86f2-ca97d2ddb260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:18.821282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab74c4-9be9-441e-86f2-ca97d2ddb260 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.821574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab74c4-9be9-441e-86f2-ca97d2ddb260 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.821858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab74c4-9be9-441e-86f2-ca97d2ddb260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.827542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01aa3f-c3c8-4c39-853a-13c344bcbda7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:18.889633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3f1f6-09d1-454d-85bd-4870de5632c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:18.890425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3f1f6-09d1-454d-85bd-4870de5632c9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:18.890717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3f1f6-09d1-454d-85bd-4870de5632c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:18.890994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3f1f6-09d1-454d-85bd-4870de5632c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:18.891785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3f1f6-09d1-454d-85bd-4870de5632c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:18.892093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3f1f6-09d1-454d-85bd-4870de5632c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:18.893774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3f1f6-09d1-454d-85bd-4870de5632c9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:18.913698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 58/113] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:18.914498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3f1f6-09d1-454d-85bd-4870de5632c9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:18.916705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:18.958034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab74c4-9be9-441e-86f2-ca97d2ddb260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:18.959624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab74c4-9be9-441e-86f2-ca97d2ddb260 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:19.031769 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:19.110617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 56/114] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:19.112092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab74c4-9be9-441e-86f2-ca97d2ddb260 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:19.126631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3f1f6-09d1-454d-85bd-4870de5632c9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:19.168112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e91c3-390b-489b-a673-589615a4009d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:19.168796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e91c3-390b-489b-a673-589615a4009d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.169102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e91c3-390b-489b-a673-589615a4009d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.169393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e91c3-390b-489b-a673-589615a4009d None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.170206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e91c3-390b-489b-a673-589615a4009d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:19.170548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e91c3-390b-489b-a673-589615a4009d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:19.172088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e91c3-390b-489b-a673-589615a4009d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:19.181302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 59/115] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:19.182166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e91c3-390b-489b-a673-589615a4009d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:19.203276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:19.230765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab74c4-9be9-441e-86f2-ca97d2ddb260 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:19.283855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190949ed-6e82-42f0-b2b8-95792ab6e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:19.285188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190949ed-6e82-42f0-b2b8-95792ab6e009 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.285493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190949ed-6e82-42f0-b2b8-95792ab6e009 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.285781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190949ed-6e82-42f0-b2b8-95792ab6e009 None admin] PATH_INFO: `/v3/users/6e889b4f5d384fef831673b867e27bfa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.287169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190949ed-6e82-42f0-b2b8-95792ab6e009 None admin] RBAC: Authorizing `identity:get_user(user_id=6e889b4f5d384fef831673b867e27bfa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:19.288689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190949ed-6e82-42f0-b2b8-95792ab6e009 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:19.290371 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 57/116] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:19 2022] GET /identity/v3/users/6e889b4f5d384fef831673b867e27bfa => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:19.291620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdc65a-49af-4487-a82e-e2107f21dce5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.291910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdc65a-49af-4487-a82e-e2107f21dce5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.292192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdc65a-49af-4487-a82e-e2107f21dce5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.293595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 58/117] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:19.294181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdc65a-49af-4487-a82e-e2107f21dce5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:19.304365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e91c3-390b-489b-a673-589615a4009d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:19.324261 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:19.343331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d123d-0a5c-42ba-ba11-989410c951c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:19.344334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d123d-0a5c-42ba-ba11-989410c951c9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.344635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d123d-0a5c-42ba-ba11-989410c951c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.344916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d123d-0a5c-42ba-ba11-989410c951c9 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.346236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d123d-0a5c-42ba-ba11-989410c951c9 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:19.347730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d123d-0a5c-42ba-ba11-989410c951c9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:19.354451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 60/118] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:19 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:19.355590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e2c2e-0e32-47bb-ad43-8e4cbba1e186 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.355893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e2c2e-0e32-47bb-ad43-8e4cbba1e186 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.356185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e2c2e-0e32-47bb-ad43-8e4cbba1e186 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.361743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 61/119] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:19.361743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e2c2e-0e32-47bb-ad43-8e4cbba1e186 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:19.435544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdc65a-49af-4487-a82e-e2107f21dce5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:19.473230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e2c2e-0e32-47bb-ad43-8e4cbba1e186 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:19.484439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2aafff-4fe5-47da-be6a-273c2e9c1761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:19.485639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2aafff-4fe5-47da-be6a-273c2e9c1761 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.485934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2aafff-4fe5-47da-be6a-273c2e9c1761 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.486392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2aafff-4fe5-47da-be6a-273c2e9c1761 None admin] PATH_INFO: `/v3/users/9a8c4f38edad4724ac733753692dda22` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.494096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2aafff-4fe5-47da-be6a-273c2e9c1761 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8c4f38edad4724ac733753692dda22)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:19.494096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2aafff-4fe5-47da-be6a-273c2e9c1761 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:19.494096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 59/120] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:19 2022] GET /identity/v3/users/9a8c4f38edad4724ac733753692dda22 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:19.494096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53b11d-0814-4a2e-9bab-111fb835bcc8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.494096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53b11d-0814-4a2e-9bab-111fb835bcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.494096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53b11d-0814-4a2e-9bab-111fb835bcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.515745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ce53b11d-0814-4a2e-9bab-111fb835bcc8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:19.516855 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ce844-e7d4-4984-8960-3cde52afd220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:19.516977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce53b11d-0814-4a2e-9bab-111fb835bcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:19.517733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ce844-e7d4-4984-8960-3cde52afd220 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.518022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ce844-e7d4-4984-8960-3cde52afd220 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.518304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ce844-e7d4-4984-8960-3cde52afd220 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.519512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ce844-e7d4-4984-8960-3cde52afd220 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:19.520969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ce844-e7d4-4984-8960-3cde52afd220 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:19.522854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 62/121] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:19 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:19.525979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:19.528520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ce844-e7d4-4984-8960-3cde52afd220 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:19.586955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:19.595027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 60/122] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:19.596230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bf6db-c136-444d-b508-d74151c16dba None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.596645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bf6db-c136-444d-b508-d74151c16dba None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.597027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bf6db-c136-444d-b508-d74151c16dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.614832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-df5bf6db-c136-444d-b508-d74151c16dba None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:19.616406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-df5bf6db-c136-444d-b508-d74151c16dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:19.660644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ce844-e7d4-4984-8960-3cde52afd220 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:19.688150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 61/123] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:19.688762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bf6db-c136-444d-b508-d74151c16dba None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:19.705492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebe810-f02b-4504-9f7d-d81ba2d81ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:19.706824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebe810-f02b-4504-9f7d-d81ba2d81ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.707687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebe810-f02b-4504-9f7d-d81ba2d81ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.708269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebe810-f02b-4504-9f7d-d81ba2d81ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.709560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebe810-f02b-4504-9f7d-d81ba2d81ec7 None admin] RBAC: Adding query filter params (role.id=a5a3836660244b6eb47bc34d9d97907a, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701, user.id=06f60bd270b04444b3c9d9fe277097bc) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:19.709883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebe810-f02b-4504-9f7d-d81ba2d81ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:19.711897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebe810-f02b-4504-9f7d-d81ba2d81ec7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:19.717945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 63/124] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:19 2022] GET /identity/v3/role_assignments?user.id=06f60bd270b04444b3c9d9fe277097bc&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=a5a3836660244b6eb47bc34d9d97907a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:19.718601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebe810-f02b-4504-9f7d-d81ba2d81ec7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:19.784326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:19.830638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bf6db-c136-444d-b508-d74151c16dba None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:19.842359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebe810-f02b-4504-9f7d-d81ba2d81ec7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:19.869023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:19.872572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0d286-9620-4277-9730-1625096ae9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:19.874200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0d286-9620-4277-9730-1625096ae9ce None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.874748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0d286-9620-4277-9730-1625096ae9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.875317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0d286-9620-4277-9730-1625096ae9ce None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.876939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0d286-9620-4277-9730-1625096ae9ce None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:19.878536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0d286-9620-4277-9730-1625096ae9ce None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:19.883318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 62/125] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:19 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:19.883318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0d286-9620-4277-9730-1625096ae9ce None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:19.901679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516756b-f635-4fb9-a22b-26cdad5f705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:19.902597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516756b-f635-4fb9-a22b-26cdad5f705c None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:19.902892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516756b-f635-4fb9-a22b-26cdad5f705c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:19.903169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516756b-f635-4fb9-a22b-26cdad5f705c None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/users/6e889b4f5d384fef831673b867e27bfa/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:19.905268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516756b-f635-4fb9-a22b-26cdad5f705c None admin] RBAC: Authorizing `identity:create_grant(project_id=c49e9ae4e8bc4168a9cab3aef4029701, user_id=6e889b4f5d384fef831673b867e27bfa, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:19.906726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516756b-f635-4fb9-a22b-26cdad5f705c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:19.926450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 64/126] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:00:19 2022] PUT /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/users/6e889b4f5d384fef831673b867e27bfa/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:19.927424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516756b-f635-4fb9-a22b-26cdad5f705c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:20.019084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0d286-9620-4277-9730-1625096ae9ce None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:20.064477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e50e63-0edd-4fb6-9da3-ffa2a923d5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:20.065315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e50e63-0edd-4fb6-9da3-ffa2a923d5bd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:20.065611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e50e63-0edd-4fb6-9da3-ffa2a923d5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:20.065899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e50e63-0edd-4fb6-9da3-ffa2a923d5bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:20.075741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:20.081228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e50e63-0edd-4fb6-9da3-ffa2a923d5bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-41e50e63-0edd-4fb6-9da3-ffa2a923d5bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:20.093410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:20.095124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 63/127] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:20.096718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e50e63-0edd-4fb6-9da3-ffa2a923d5bd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:20.126847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516756b-f635-4fb9-a22b-26cdad5f705c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:20.218837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7a88a-d52e-4f12-8dd9-944bd07e78be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:20.219767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7a88a-d52e-4f12-8dd9-944bd07e78be None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:20.220034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7a88a-d52e-4f12-8dd9-944bd07e78be None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:20.220316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7a88a-d52e-4f12-8dd9-944bd07e78be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:20.222144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e50e63-0edd-4fb6-9da3-ffa2a923d5bd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:20.244406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7a88a-d52e-4f12-8dd9-944bd07e78be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d2c7a88a-d52e-4f12-8dd9-944bd07e78be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:20.260522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:20.264137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:20.267444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 65/128] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:20.277368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0693e-fd92-4656-b254-42db21648ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:20.277641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7a88a-d52e-4f12-8dd9-944bd07e78be None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:20.277837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0693e-fd92-4656-b254-42db21648ede None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:20.277837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0693e-fd92-4656-b254-42db21648ede None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:20.277837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0693e-fd92-4656-b254-42db21648ede None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/users/9a8c4f38edad4724ac733753692dda22/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:20.277837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0693e-fd92-4656-b254-42db21648ede None admin] RBAC: Authorizing `identity:create_grant(project_id=c49e9ae4e8bc4168a9cab3aef4029701, user_id=9a8c4f38edad4724ac733753692dda22, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:20.277837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0693e-fd92-4656-b254-42db21648ede None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:20.294181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 64/129] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:00:20 2022] PUT /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/users/9a8c4f38edad4724ac733753692dda22/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:20.294750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0693e-fd92-4656-b254-42db21648ede None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:20.360280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:20.414106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:20.448327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7a88a-d52e-4f12-8dd9-944bd07e78be None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:20.492891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0693e-fd92-4656-b254-42db21648ede None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:20.496616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84044b53-f317-4a66-bcf3-443d98202aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:20.497302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84044b53-f317-4a66-bcf3-443d98202aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:20.497590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84044b53-f317-4a66-bcf3-443d98202aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:20.497869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84044b53-f317-4a66-bcf3-443d98202aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:20.498681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84044b53-f317-4a66-bcf3-443d98202aa4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:20.498990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84044b53-f317-4a66-bcf3-443d98202aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:20.500424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84044b53-f317-4a66-bcf3-443d98202aa4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:20.511670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 66/130] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:20.512457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d7668-8204-4739-8121-ba7084203d31 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:20.512752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d7668-8204-4739-8121-ba7084203d31 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:20.513036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d7668-8204-4739-8121-ba7084203d31 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:20.515025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 67/131] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:20.520958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cfa10-2da6-4f83-a2fb-595ae10d9635 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:20.521251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cfa10-2da6-4f83-a2fb-595ae10d9635 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:20.521538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cfa10-2da6-4f83-a2fb-595ae10d9635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:20.542368 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-359cfa10-2da6-4f83-a2fb-595ae10d9635 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:20.544154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-359cfa10-2da6-4f83-a2fb-595ae10d9635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:20.546028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee0ee4-76b4-4229-80ec-5cd85db7a0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:20.547713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee0ee4-76b4-4229-80ec-5cd85db7a0bb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:20.548016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee0ee4-76b4-4229-80ec-5cd85db7a0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:20.548296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee0ee4-76b4-4229-80ec-5cd85db7a0bb None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:20.549073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee0ee4-76b4-4229-80ec-5cd85db7a0bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:20.555337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee0ee4-76b4-4229-80ec-5cd85db7a0bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:20.555337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee0ee4-76b4-4229-80ec-5cd85db7a0bb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:20.562889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 65/132] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:20.563908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee0ee4-76b4-4229-80ec-5cd85db7a0bb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:20.629055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 68/133] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:20.629939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cfa10-2da6-4f83-a2fb-595ae10d9635 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:20.666132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:20.770995 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee0ee4-76b4-4229-80ec-5cd85db7a0bb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:20.805625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cfa10-2da6-4f83-a2fb-595ae10d9635 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:20.808760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:20.838980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaef5c-0a1c-46aa-9c7a-4a4d3b2f5f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:20.839974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaef5c-0a1c-46aa-9c7a-4a4d3b2f5f45 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:20.840249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaef5c-0a1c-46aa-9c7a-4a4d3b2f5f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:20.840530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaef5c-0a1c-46aa-9c7a-4a4d3b2f5f45 None admin] PATH_INFO: `/v3/users/6e889b4f5d384fef831673b867e27bfa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:20.842558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaef5c-0a1c-46aa-9c7a-4a4d3b2f5f45 None admin] RBAC: Authorizing `identity:get_user(user_id=6e889b4f5d384fef831673b867e27bfa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:20.844144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaef5c-0a1c-46aa-9c7a-4a4d3b2f5f45 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:20.845817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 66/134] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:20 2022] GET /identity/v3/users/6e889b4f5d384fef831673b867e27bfa => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:20.846591 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaef5c-0a1c-46aa-9c7a-4a4d3b2f5f45 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:20.864198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43205-e2b5-4de1-aa66-f42246358a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:20.865353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43205-e2b5-4de1-aa66-f42246358a50 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:20.865648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43205-e2b5-4de1-aa66-f42246358a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:20.865915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43205-e2b5-4de1-aa66-f42246358a50 None admin] PATH_INFO: `/v3/users/6e889b4f5d384fef831673b867e27bfa` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:20.872171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe43205-e2b5-4de1-aa66-f42246358a50 None admin] RBAC: Authorizing `identity:get_user(user_id=6e889b4f5d384fef831673b867e27bfa)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:20.875170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe43205-e2b5-4de1-aa66-f42246358a50 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:20.880221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 69/135] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:20 2022] GET /identity/v3/users/6e889b4f5d384fef831673b867e27bfa => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:20.881120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43205-e2b5-4de1-aa66-f42246358a50 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:21.052402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaef5c-0a1c-46aa-9c7a-4a4d3b2f5f45 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:21.080477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:21.087344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43205-e2b5-4de1-aa66-f42246358a50 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:21.130479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef677b55-b803-47c4-b3ca-d3c43d32580d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:21.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef677b55-b803-47c4-b3ca-d3c43d32580d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:21.132672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef677b55-b803-47c4-b3ca-d3c43d32580d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:21.133039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef677b55-b803-47c4-b3ca-d3c43d32580d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:21.134813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef677b55-b803-47c4-b3ca-d3c43d32580d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:21.136338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef677b55-b803-47c4-b3ca-d3c43d32580d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:21.139573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 67/136] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:21.146314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef677b55-b803-47c4-b3ca-d3c43d32580d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:21.153297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594442ae-8910-410a-850d-242f419ae9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:21.154246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594442ae-8910-410a-850d-242f419ae9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:21.154573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594442ae-8910-410a-850d-242f419ae9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:21.154907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594442ae-8910-410a-850d-242f419ae9d0 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:21.156226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594442ae-8910-410a-850d-242f419ae9d0 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:21.157685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594442ae-8910-410a-850d-242f419ae9d0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:21.160229 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 70/137] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:20 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:21.160494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594442ae-8910-410a-850d-242f419ae9d0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:21.255428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:21.338317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594442ae-8910-410a-850d-242f419ae9d0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:21.352538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef677b55-b803-47c4-b3ca-d3c43d32580d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:21.388553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb13e74-5225-4869-b545-c768a1816fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:21.388553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb13e74-5225-4869-b545-c768a1816fc9 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:21.388553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb13e74-5225-4869-b545-c768a1816fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:21.388553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb13e74-5225-4869-b545-c768a1816fc9 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:21.390418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb13e74-5225-4869-b545-c768a1816fc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:21.390418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb13e74-5225-4869-b545-c768a1816fc9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:21.425353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d4d60-7c38-4bae-9cb9-db21950a70c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:21.426241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d4d60-7c38-4bae-9cb9-db21950a70c8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:21.426567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d4d60-7c38-4bae-9cb9-db21950a70c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:21.426854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d4d60-7c38-4bae-9cb9-db21950a70c8 None admin] PATH_INFO: `/v3/projects/000e1f52e4464a07aca1f72426e11112` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:21.431147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 71/138] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:00:21 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:21.432008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb13e74-5225-4869-b545-c768a1816fc9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:21.432789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d4d60-7c38-4bae-9cb9-db21950a70c8 None admin] RBAC: Authorizing `identity:get_project(project_id=000e1f52e4464a07aca1f72426e11112)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:21.434337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d4d60-7c38-4bae-9cb9-db21950a70c8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:21.437171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 68/139] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:21 2022] GET /identity/v3/projects/000e1f52e4464a07aca1f72426e11112 => generated 312 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:21.448788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d4d60-7c38-4bae-9cb9-db21950a70c8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:21.499688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:21.677554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb13e74-5225-4869-b545-c768a1816fc9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:21.727108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e724da-85c6-4090-a119-61a7bfa4f962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:21.728057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e724da-85c6-4090-a119-61a7bfa4f962 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:21.728346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e724da-85c6-4090-a119-61a7bfa4f962 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:21.728630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e724da-85c6-4090-a119-61a7bfa4f962 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/users/6e889b4f5d384fef831673b867e27bfa/roles/2e875b35d6744573ae0287b954108209` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:21.732238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e724da-85c6-4090-a119-61a7bfa4f962 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49e9ae4e8bc4168a9cab3aef4029701, user_id=6e889b4f5d384fef831673b867e27bfa, role_id=2e875b35d6744573ae0287b954108209)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:21.734284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e724da-85c6-4090-a119-61a7bfa4f962 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:21.741528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d4d60-7c38-4bae-9cb9-db21950a70c8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:21.759892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 72/140] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:00:21 2022] PUT /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/users/6e889b4f5d384fef831673b867e27bfa/roles/2e875b35d6744573ae0287b954108209 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:21.778608 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:21.788739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bbda4-0698-4c6b-9e41-6b8b6a649ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:21.789666 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbda4-0698-4c6b-9e41-6b8b6a649ee6 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:21.789955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbda4-0698-4c6b-9e41-6b8b6a649ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:21.790244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbda4-0698-4c6b-9e41-6b8b6a649ee6 None admin] PATH_INFO: `/v3/projects/000e1f52e4464a07aca1f72426e11112/users/6e889b4f5d384fef831673b867e27bfa/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:21.792343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bbda4-0698-4c6b-9e41-6b8b6a649ee6 None admin] RBAC: Authorizing `identity:create_grant(project_id=000e1f52e4464a07aca1f72426e11112, user_id=6e889b4f5d384fef831673b867e27bfa, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:21.793836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bbda4-0698-4c6b-9e41-6b8b6a649ee6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:21.815748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 69/141] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:00:21 2022] PUT /identity/v3/projects/000e1f52e4464a07aca1f72426e11112/users/6e889b4f5d384fef831673b867e27bfa/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:22.660075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ba60-eb08-44ec-8f14-6f142ed5c6b9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:22.660370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ba60-eb08-44ec-8f14-6f142ed5c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:22.660651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ba60-eb08-44ec-8f14-6f142ed5c6b9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:22.662207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 73/142] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:22.667763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f83d8-edde-4f9b-a70c-6307af3532d9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:22.668057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f83d8-edde-4f9b-a70c-6307af3532d9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:22.668374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f83d8-edde-4f9b-a70c-6307af3532d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:22.687598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2c3f83d8-edde-4f9b-a70c-6307af3532d9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:22.688800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c3f83d8-edde-4f9b-a70c-6307af3532d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:22.782287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 70/143] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:22.818813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6ba60-eb08-44ec-8f14-6f142ed5c6b9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:22.930848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:22.955028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6ba60-eb08-44ec-8f14-6f142ed5c6b9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:23.001115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeaecf-28ed-4ca1-ad18-d72437298041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:23.001945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeaecf-28ed-4ca1-ad18-d72437298041 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:23.002239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeaecf-28ed-4ca1-ad18-d72437298041 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:23.002533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeaecf-28ed-4ca1-ad18-d72437298041 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:23.017123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbeaecf-28ed-4ca1-ad18-d72437298041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fcbeaecf-28ed-4ca1-ad18-d72437298041 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:23.024981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:23.025863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:23.026673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 74/144] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:23.032702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f83d8-edde-4f9b-a70c-6307af3532d9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:23.177802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f83d8-edde-4f9b-a70c-6307af3532d9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:23.201951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:23.223725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d4f51-3fe7-40c3-9d76-e0a75a1ede56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:23.224408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d4f51-3fe7-40c3-9d76-e0a75a1ede56 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:23.224711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d4f51-3fe7-40c3-9d76-e0a75a1ede56 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:23.224978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d4f51-3fe7-40c3-9d76-e0a75a1ede56 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:23.225748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d4f51-3fe7-40c3-9d76-e0a75a1ede56 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:23.226059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d4f51-3fe7-40c3-9d76-e0a75a1ede56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:23.227474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d4f51-3fe7-40c3-9d76-e0a75a1ede56 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:23.239941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 71/145] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:23.245345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeaecf-28ed-4ca1-ad18-d72437298041 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:23.330403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d9d68-b5e4-4321-a63b-b867d95241c2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:23.330754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d9d68-b5e4-4321-a63b-b867d95241c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:23.331040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d9d68-b5e4-4321-a63b-b867d95241c2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:23.332708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 72/146] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:23.337745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778910e-64c5-4408-9e97-21304dfb9add None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:23.338038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778910e-64c5-4408-9e97-21304dfb9add None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:23.338320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778910e-64c5-4408-9e97-21304dfb9add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:23.369221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9778910e-64c5-4408-9e97-21304dfb9add None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:23.370419 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9778910e-64c5-4408-9e97-21304dfb9add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:23.420865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeaecf-28ed-4ca1-ad18-d72437298041 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:23.516176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31351a-9034-4f6f-82b2-b3d830091570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:23.517032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31351a-9034-4f6f-82b2-b3d830091570 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:23.517594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31351a-9034-4f6f-82b2-b3d830091570 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:23.517885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31351a-9034-4f6f-82b2-b3d830091570 None admin] PATH_INFO: `/v3/users/6e889b4f5d384fef831673b867e27bfa` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:23.519215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31351a-9034-4f6f-82b2-b3d830091570 None admin] RBAC: Authorizing `identity:get_user(user_id=6e889b4f5d384fef831673b867e27bfa)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:23.520766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31351a-9034-4f6f-82b2-b3d830091570 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:23.522657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 75/147] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:23 2022] GET /identity/v3/users/6e889b4f5d384fef831673b867e27bfa => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:23.524639 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91151a41-6af3-4f0d-80eb-3c085a56777e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:23.524948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91151a41-6af3-4f0d-80eb-3c085a56777e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:23.525237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91151a41-6af3-4f0d-80eb-3c085a56777e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:23.533071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 76/148] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:23.533071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91151a41-6af3-4f0d-80eb-3c085a56777e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:23.553511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:23.572650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 73/149] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:23.574033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce5d6b-2523-4edd-aae0-390a65ea97b4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:23.574326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce5d6b-2523-4edd-aae0-390a65ea97b4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:23.574611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce5d6b-2523-4edd-aae0-390a65ea97b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:23.615929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6fce5d6b-2523-4edd-aae0-390a65ea97b4 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:23.617102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6fce5d6b-2523-4edd-aae0-390a65ea97b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:23.712181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91151a41-6af3-4f0d-80eb-3c085a56777e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:23.727299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 74/150] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:23.727299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce5d6b-2523-4edd-aae0-390a65ea97b4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:23.768486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1653b-1a15-4527-99d7-70516872e761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:23.769491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1653b-1a15-4527-99d7-70516872e761 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:23.769987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1653b-1a15-4527-99d7-70516872e761 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:23.770470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1653b-1a15-4527-99d7-70516872e761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:23.782855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:23.859027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce5d6b-2523-4edd-aae0-390a65ea97b4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:23.881017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1653b-1a15-4527-99d7-70516872e761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:23.882600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1653b-1a15-4527-99d7-70516872e761 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:23.897376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587876c-cf30-47de-865b-a900773589df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:23.898460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587876c-cf30-47de-865b-a900773589df None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:23.898958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587876c-cf30-47de-865b-a900773589df None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:23.899458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587876c-cf30-47de-865b-a900773589df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:23.900103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:23.930078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587876c-cf30-47de-865b-a900773589df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7587876c-cf30-47de-865b-a900773589df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:23.939162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:23.940086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:23.940924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:23.949696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 75/151] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:23.951452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587876c-cf30-47de-865b-a900773589df None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:24.089552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:24.094098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587876c-cf30-47de-865b-a900773589df None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:24.121934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 77/152] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:24.123680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1653b-1a15-4527-99d7-70516872e761 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:24.141147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444f8b7-7da6-4d8e-8f16-45cb332190ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:24.142606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444f8b7-7da6-4d8e-8f16-45cb332190ad None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:24.143074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444f8b7-7da6-4d8e-8f16-45cb332190ad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:24.143551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444f8b7-7da6-4d8e-8f16-45cb332190ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:24.155506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444f8b7-7da6-4d8e-8f16-45cb332190ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b444f8b7-7da6-4d8e-8f16-45cb332190ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:24.166046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:24.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:24.167694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:24.170333 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 76/153] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:24.171306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444f8b7-7da6-4d8e-8f16-45cb332190ad None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:24.276661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444f8b7-7da6-4d8e-8f16-45cb332190ad None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:24.317053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300dc16-1908-4af6-8931-f1ef066c815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:24.318302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300dc16-1908-4af6-8931-f1ef066c815b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:24.318775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300dc16-1908-4af6-8931-f1ef066c815b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:24.319236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300dc16-1908-4af6-8931-f1ef066c815b None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:24.320896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300dc16-1908-4af6-8931-f1ef066c815b None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:24.322590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300dc16-1908-4af6-8931-f1ef066c815b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:24.323349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1653b-1a15-4527-99d7-70516872e761 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:24.325179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 77/154] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:24 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:24.326682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300dc16-1908-4af6-8931-f1ef066c815b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:24.342719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:24.371650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:24.387026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fd7de-ae81-42a4-8be6-73dd5b112099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:24.387766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fd7de-ae81-42a4-8be6-73dd5b112099 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:24.388058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fd7de-ae81-42a4-8be6-73dd5b112099 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:24.388344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fd7de-ae81-42a4-8be6-73dd5b112099 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:24.389772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fd7de-ae81-42a4-8be6-73dd5b112099 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:24.390091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fd7de-ae81-42a4-8be6-73dd5b112099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:24.391533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fd7de-ae81-42a4-8be6-73dd5b112099 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:24.413341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 78/155] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:24.414386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fd7de-ae81-42a4-8be6-73dd5b112099 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:24.457425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300dc16-1908-4af6-8931-f1ef066c815b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:24.497210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9da32c-821a-45f1-b6f3-44bc452c84b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:24.498810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9da32c-821a-45f1-b6f3-44bc452c84b8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:24.499359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9da32c-821a-45f1-b6f3-44bc452c84b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:24.499839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9da32c-821a-45f1-b6f3-44bc452c84b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:24.500827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9da32c-821a-45f1-b6f3-44bc452c84b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:24.501359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9da32c-821a-45f1-b6f3-44bc452c84b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:24.503091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9da32c-821a-45f1-b6f3-44bc452c84b8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:24.513031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 78/156] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:24.514841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9da32c-821a-45f1-b6f3-44bc452c84b8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:24.597680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fd7de-ae81-42a4-8be6-73dd5b112099 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:24.639680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:24.647912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9da32c-821a-45f1-b6f3-44bc452c84b8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:24.655412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bf310-5a3a-4a81-862d-6abb54b36afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:24.656092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bf310-5a3a-4a81-862d-6abb54b36afd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:24.656383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bf310-5a3a-4a81-862d-6abb54b36afd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:24.656667 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bf310-5a3a-4a81-862d-6abb54b36afd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:24.657530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bf310-5a3a-4a81-862d-6abb54b36afd None admin] RBAC: Adding query filter params (role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701, user.id=6e889b4f5d384fef831673b867e27bfa) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:24.657848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bf310-5a3a-4a81-862d-6abb54b36afd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:24.663704 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bf310-5a3a-4a81-862d-6abb54b36afd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:24.679287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 79/157] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:24 2022] GET /identity/v3/role_assignments?user.id=6e889b4f5d384fef831673b867e27bfa&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:24.679287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bf310-5a3a-4a81-862d-6abb54b36afd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:24.711332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49e26b-b92c-4757-aded-c1ea7a2a2006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:24.712622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49e26b-b92c-4757-aded-c1ea7a2a2006 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:24.713112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49e26b-b92c-4757-aded-c1ea7a2a2006 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:24.713620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49e26b-b92c-4757-aded-c1ea7a2a2006 None admin] PATH_INFO: `/v3/users/06f60bd270b04444b3c9d9fe277097bc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:24.720272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49e26b-b92c-4757-aded-c1ea7a2a2006 None admin] RBAC: Authorizing `identity:get_user(user_id=06f60bd270b04444b3c9d9fe277097bc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:24.722359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49e26b-b92c-4757-aded-c1ea7a2a2006 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:24.724750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 79/158] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:24 2022] GET /identity/v3/users/06f60bd270b04444b3c9d9fe277097bc => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:24.733437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49e26b-b92c-4757-aded-c1ea7a2a2006 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:24.742860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:24.843242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bf310-5a3a-4a81-862d-6abb54b36afd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:24.892060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49e26b-b92c-4757-aded-c1ea7a2a2006 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:24.900964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31463bc6-8313-49c9-9ca2-d283cea1def7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:24.901965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31463bc6-8313-49c9-9ca2-d283cea1def7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:24.902333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31463bc6-8313-49c9-9ca2-d283cea1def7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:24.902695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31463bc6-8313-49c9-9ca2-d283cea1def7 None admin] PATH_INFO: `/v3/users/9a8c4f38edad4724ac733753692dda22` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:24.904021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31463bc6-8313-49c9-9ca2-d283cea1def7 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8c4f38edad4724ac733753692dda22)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:24.906454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31463bc6-8313-49c9-9ca2-d283cea1def7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:24.912410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 80/159] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:24 2022] GET /identity/v3/users/9a8c4f38edad4724ac733753692dda22 => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:24.924284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31463bc6-8313-49c9-9ca2-d283cea1def7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:24.957014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fc45e-0305-45a1-9b26-c0d137d73d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:24.958859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fc45e-0305-45a1-9b26-c0d137d73d95 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:24.959655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fc45e-0305-45a1-9b26-c0d137d73d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:24.960381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fc45e-0305-45a1-9b26-c0d137d73d95 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:24.964494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fc45e-0305-45a1-9b26-c0d137d73d95 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:24.968106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fc45e-0305-45a1-9b26-c0d137d73d95 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:24.975944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 80/160] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:24 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:25.004725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fc45e-0305-45a1-9b26-c0d137d73d95 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:25.051686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:25.106267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31463bc6-8313-49c9-9ca2-d283cea1def7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:25.112774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:25.173296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803cf36-0aa8-4344-8bf8-6726e3ae2121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:25.174299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803cf36-0aa8-4344-8bf8-6726e3ae2121 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:25.174847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803cf36-0aa8-4344-8bf8-6726e3ae2121 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:25.175468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803cf36-0aa8-4344-8bf8-6726e3ae2121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:25.181062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fc45e-0305-45a1-9b26-c0d137d73d95 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:25.220362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42410f0-9166-47ac-8daf-79133d614879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:25.221503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42410f0-9166-47ac-8daf-79133d614879 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:25.221936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42410f0-9166-47ac-8daf-79133d614879 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:25.222387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42410f0-9166-47ac-8daf-79133d614879 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/users/06f60bd270b04444b3c9d9fe277097bc/roles/a5a3836660244b6eb47bc34d9d97907a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:25.224578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42410f0-9166-47ac-8daf-79133d614879 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49e9ae4e8bc4168a9cab3aef4029701, user_id=06f60bd270b04444b3c9d9fe277097bc, role_id=a5a3836660244b6eb47bc34d9d97907a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:25.226197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42410f0-9166-47ac-8daf-79133d614879 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:25.247230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 81/161] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:00:25 2022] PUT /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/users/06f60bd270b04444b3c9d9fe277097bc/roles/a5a3836660244b6eb47bc34d9d97907a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:25.291493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803cf36-0aa8-4344-8bf8-6726e3ae2121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:25.293294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803cf36-0aa8-4344-8bf8-6726e3ae2121 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:25.477506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 81/162] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:25.483766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42410f0-9166-47ac-8daf-79133d614879 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:25.620972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:25.656029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42410f0-9166-47ac-8daf-79133d614879 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:25.722497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcef6f-7d42-47a9-8d58-eeef384fb452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:25.724470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcef6f-7d42-47a9-8d58-eeef384fb452 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:25.725180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcef6f-7d42-47a9-8d58-eeef384fb452 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:25.725872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcef6f-7d42-47a9-8d58-eeef384fb452 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:25.733175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcef6f-7d42-47a9-8d58-eeef384fb452 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:25.736372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcef6f-7d42-47a9-8d58-eeef384fb452 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:25.740284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 82/163] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:25 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:25.747540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803cf36-0aa8-4344-8bf8-6726e3ae2121 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:25.757099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:25.917193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803cf36-0aa8-4344-8bf8-6726e3ae2121 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:25.965366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c01e4-db7e-4931-ac9b-df3f87761b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:25.966044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c01e4-db7e-4931-ac9b-df3f87761b2e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:25.966338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c01e4-db7e-4931-ac9b-df3f87761b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:25.967160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c01e4-db7e-4931-ac9b-df3f87761b2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:25.968050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c01e4-db7e-4931-ac9b-df3f87761b2e None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701, user.id=9a8c4f38edad4724ac733753692dda22) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:25.968370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c01e4-db7e-4931-ac9b-df3f87761b2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:25.969834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c01e4-db7e-4931-ac9b-df3f87761b2e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:25.979667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de6d7f-73b6-48e9-84d2-f4e4b96022a6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:25.980177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de6d7f-73b6-48e9-84d2-f4e4b96022a6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:25.980252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 82/164] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:25 2022] GET /identity/v3/role_assignments?user.id=9a8c4f38edad4724ac733753692dda22&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:25.980338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de6d7f-73b6-48e9-84d2-f4e4b96022a6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:25.981930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 83/165] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:25.988909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410a7c2-c1bd-41b8-8148-b85e2b0160ba None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:25.989215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410a7c2-c1bd-41b8-8148-b85e2b0160ba None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:25.989536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410a7c2-c1bd-41b8-8148-b85e2b0160ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:26.016818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6410a7c2-c1bd-41b8-8148-b85e2b0160ba None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:26.018049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6410a7c2-c1bd-41b8-8148-b85e2b0160ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:26.106373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:26.117947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 83/166] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:26.171417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de6d7f-73b6-48e9-84d2-f4e4b96022a6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:26.312138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de6d7f-73b6-48e9-84d2-f4e4b96022a6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:26.353857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b332f1c-eaa6-4f05-9f2b-51e4554f3e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:26.355024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b332f1c-eaa6-4f05-9f2b-51e4554f3e21 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:26.355539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b332f1c-eaa6-4f05-9f2b-51e4554f3e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:26.356017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b332f1c-eaa6-4f05-9f2b-51e4554f3e21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:26.372884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b332f1c-eaa6-4f05-9f2b-51e4554f3e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0b332f1c-eaa6-4f05-9f2b-51e4554f3e21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:26.381138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:26.382003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:26.383059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:26.389511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63ad63-6f0c-4c5d-9315-8eabd3336013 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:26.389818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63ad63-6f0c-4c5d-9315-8eabd3336013 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:26.390141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63ad63-6f0c-4c5d-9315-8eabd3336013 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:26.390297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 84/167] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:26.391532 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 84/168] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:26.395827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b332f1c-eaa6-4f05-9f2b-51e4554f3e21 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:26.407586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2c2dd-2eee-47c1-b89d-b67c64d37503 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:26.407886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2c2dd-2eee-47c1-b89d-b67c64d37503 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:26.408178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2c2dd-2eee-47c1-b89d-b67c64d37503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:26.418686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:26.440423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a5d2c2dd-2eee-47c1-b89d-b67c64d37503 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:26.441882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a5d2c2dd-2eee-47c1-b89d-b67c64d37503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:26.535861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b332f1c-eaa6-4f05-9f2b-51e4554f3e21 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:26.564580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 85/169] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:26.577555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9b0ba-c14a-4609-84cc-0e5c7525d3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:26.578241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9b0ba-c14a-4609-84cc-0e5c7525d3bf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:26.578528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9b0ba-c14a-4609-84cc-0e5c7525d3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:26.578810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9b0ba-c14a-4609-84cc-0e5c7525d3bf None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:26.579605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9b0ba-c14a-4609-84cc-0e5c7525d3bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:26.579918 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9b0ba-c14a-4609-84cc-0e5c7525d3bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:26.581328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9b0ba-c14a-4609-84cc-0e5c7525d3bf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:26.599300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2c2dd-2eee-47c1-b89d-b67c64d37503 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:26.600628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 85/170] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:26.610066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9b0ba-c14a-4609-84cc-0e5c7525d3bf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:26.704442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:26.737165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2c2dd-2eee-47c1-b89d-b67c64d37503 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:26.752428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9b0ba-c14a-4609-84cc-0e5c7525d3bf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:26.785030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:26.806700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f8733-ad65-4bc3-8866-14d0d3f5fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:26.807824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8733-ad65-4bc3-8866-14d0d3f5fcbc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:26.808344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8733-ad65-4bc3-8866-14d0d3f5fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:26.808812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8733-ad65-4bc3-8866-14d0d3f5fcbc None admin] PATH_INFO: `/v3/users/6e889b4f5d384fef831673b867e27bfa` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:26.810533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f8733-ad65-4bc3-8866-14d0d3f5fcbc None admin] RBAC: Authorizing `identity:get_user(user_id=6e889b4f5d384fef831673b867e27bfa)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:26.812662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f8733-ad65-4bc3-8866-14d0d3f5fcbc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:26.814371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 86/171] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:26 2022] GET /identity/v3/users/6e889b4f5d384fef831673b867e27bfa => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:26.815356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7368ce-5653-46b5-9a47-4050f798a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:26.815653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb67d9-6484-4891-baa1-5f5854bdd189 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:26.815947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb67d9-6484-4891-baa1-5f5854bdd189 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:26.816217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7368ce-5653-46b5-9a47-4050f798a6ef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:26.816318 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb67d9-6484-4891-baa1-5f5854bdd189 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:26.816517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7368ce-5653-46b5-9a47-4050f798a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:26.816806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7368ce-5653-46b5-9a47-4050f798a6ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:26.817494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 87/172] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:26.825381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb67d9-6484-4891-baa1-5f5854bdd189 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:26.851725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7368ce-5653-46b5-9a47-4050f798a6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9e7368ce-5653-46b5-9a47-4050f798a6ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:26.862068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:26.863005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:26.863834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 86/173] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:26.864564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9893a2f-c24f-4c8f-bcab-0664d46f16e8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:26.865644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9893a2f-c24f-4c8f-bcab-0664d46f16e8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:26.865644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9893a2f-c24f-4c8f-bcab-0664d46f16e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:26.905298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c9893a2f-c24f-4c8f-bcab-0664d46f16e8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:26.906523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c9893a2f-c24f-4c8f-bcab-0664d46f16e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:27.015000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb67d9-6484-4891-baa1-5f5854bdd189 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:27.031897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 87/174] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:27.032842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9893a2f-c24f-4c8f-bcab-0664d46f16e8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:27.086405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8314e4c-bcd2-41c7-b9b8-6e0d73ae6e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:27.093087 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8314e4c-bcd2-41c7-b9b8-6e0d73ae6e6a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.093434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8314e4c-bcd2-41c7-b9b8-6e0d73ae6e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.093727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8314e4c-bcd2-41c7-b9b8-6e0d73ae6e6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.119570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:27.162174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9893a2f-c24f-4c8f-bcab-0664d46f16e8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:27.171584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:27.181117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8314e4c-bcd2-41c7-b9b8-6e0d73ae6e6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:27.182990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8314e4c-bcd2-41c7-b9b8-6e0d73ae6e6a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:27.205275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6fa0c-958c-4a73-9f60-b5ef99a09daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:27.205949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6fa0c-958c-4a73-9f60-b5ef99a09daa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.206232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6fa0c-958c-4a73-9f60-b5ef99a09daa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.206527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6fa0c-958c-4a73-9f60-b5ef99a09daa None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.207276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6fa0c-958c-4a73-9f60-b5ef99a09daa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:27.207983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6fa0c-958c-4a73-9f60-b5ef99a09daa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:27.209072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6fa0c-958c-4a73-9f60-b5ef99a09daa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:27.218854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 88/175] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:27.224737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273ff0f-398d-49e4-b3a5-50c02a8d9e4d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.225031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273ff0f-398d-49e4-b3a5-50c02a8d9e4d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.225317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273ff0f-398d-49e4-b3a5-50c02a8d9e4d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.226645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 89/176] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:27.231581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273ff0f-398d-49e4-b3a5-50c02a8d9e4d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:27.297627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 88/177] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:27.298738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8314e4c-bcd2-41c7-b9b8-6e0d73ae6e6a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:27.348603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273ff0f-398d-49e4-b3a5-50c02a8d9e4d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:27.375801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:27.386883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f3aaf-a45f-455f-9ca3-f2fbe445326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:27.387719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f3aaf-a45f-455f-9ca3-f2fbe445326a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.388012 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f3aaf-a45f-455f-9ca3-f2fbe445326a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.388294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f3aaf-a45f-455f-9ca3-f2fbe445326a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.398349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:27.399446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f3aaf-a45f-455f-9ca3-f2fbe445326a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:27.406263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8314e4c-bcd2-41c7-b9b8-6e0d73ae6e6a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-269f3aaf-a45f-455f-9ca3-f2fbe445326a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:27.408102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:27.408967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 90/178] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:27.414658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416c262-7991-4440-b6e7-0ca9c5db24ca None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.415480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416c262-7991-4440-b6e7-0ca9c5db24ca None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.415480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416c262-7991-4440-b6e7-0ca9c5db24ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.431375 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2416c262-7991-4440-b6e7-0ca9c5db24ca None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:27.432579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2416c262-7991-4440-b6e7-0ca9c5db24ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:27.452309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee2430-fdda-4101-abec-d7cfdab275af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:27.453200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee2430-fdda-4101-abec-d7cfdab275af None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.453492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee2430-fdda-4101-abec-d7cfdab275af None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.453775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee2430-fdda-4101-abec-d7cfdab275af None admin] PATH_INFO: `/v3/users/6e889b4f5d384fef831673b867e27bfa` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.455450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee2430-fdda-4101-abec-d7cfdab275af None admin] RBAC: Authorizing `identity:get_user(user_id=6e889b4f5d384fef831673b867e27bfa)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:27.456920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee2430-fdda-4101-abec-d7cfdab275af None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:27.458918 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 89/179] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:27 2022] GET /identity/v3/users/6e889b4f5d384fef831673b867e27bfa => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:27.459823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fffa8a-b09d-4281-9761-754f0e3bc798 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.460061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fffa8a-b09d-4281-9761-754f0e3bc798 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.460352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fffa8a-b09d-4281-9761-754f0e3bc798 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.465761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 90/180] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:27.465761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fffa8a-b09d-4281-9761-754f0e3bc798 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:27.509388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 91/181] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:27.516151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416c262-7991-4440-b6e7-0ca9c5db24ca None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:27.585218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fffa8a-b09d-4281-9761-754f0e3bc798 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:27.624363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7b632-2ba8-41d4-8b78-04bff7cbd33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:27.625528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7b632-2ba8-41d4-8b78-04bff7cbd33d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.625818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7b632-2ba8-41d4-8b78-04bff7cbd33d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.626100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7b632-2ba8-41d4-8b78-04bff7cbd33d None admin] PATH_INFO: `/v3/projects/000e1f52e4464a07aca1f72426e11112` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.627297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7b632-2ba8-41d4-8b78-04bff7cbd33d None admin] RBAC: Authorizing `identity:get_project(project_id=000e1f52e4464a07aca1f72426e11112)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:27.628753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7b632-2ba8-41d4-8b78-04bff7cbd33d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:27.630621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 91/182] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:27 2022] GET /identity/v3/projects/000e1f52e4464a07aca1f72426e11112 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:27.631318 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7b632-2ba8-41d4-8b78-04bff7cbd33d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:27.640630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416c262-7991-4440-b6e7-0ca9c5db24ca None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:27.660416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:27.670377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:27.680755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c972e-ac41-4084-aeb5-861eb7fca529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:27.681442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c972e-ac41-4084-aeb5-861eb7fca529 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.681722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c972e-ac41-4084-aeb5-861eb7fca529 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.682004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c972e-ac41-4084-aeb5-861eb7fca529 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.682763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c972e-ac41-4084-aeb5-861eb7fca529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:27.683314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c972e-ac41-4084-aeb5-861eb7fca529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:27.684489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c972e-ac41-4084-aeb5-861eb7fca529 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:27.693891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 92/183] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:27.699480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e66610-879e-4e0b-afac-a5dd15fde96a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.699776 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e66610-879e-4e0b-afac-a5dd15fde96a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.700056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e66610-879e-4e0b-afac-a5dd15fde96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.717202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-70e66610-879e-4e0b-afac-a5dd15fde96a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:27.718404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-70e66610-879e-4e0b-afac-a5dd15fde96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:27.744788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7b632-2ba8-41d4-8b78-04bff7cbd33d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:27.783144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d97a8e-ceef-4cc8-8b6d-77fff338449f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:27.783829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d97a8e-ceef-4cc8-8b6d-77fff338449f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.784117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d97a8e-ceef-4cc8-8b6d-77fff338449f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.784399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d97a8e-ceef-4cc8-8b6d-77fff338449f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:27.801699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 93/184] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:27.802974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e66610-879e-4e0b-afac-a5dd15fde96a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:27.869917 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d97a8e-ceef-4cc8-8b6d-77fff338449f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:27.872659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d97a8e-ceef-4cc8-8b6d-77fff338449f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:27.925447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:27.953001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e66610-879e-4e0b-afac-a5dd15fde96a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:27.965816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:27.992618 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535563ce-857f-4a9e-839c-83794518bf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:27.993430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535563ce-857f-4a9e-839c-83794518bf77 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:27.993715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535563ce-857f-4a9e-839c-83794518bf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:27.993989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535563ce-857f-4a9e-839c-83794518bf77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.005721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 92/185] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:28.006806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d97a8e-ceef-4cc8-8b6d-77fff338449f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.010443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535563ce-857f-4a9e-839c-83794518bf77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-535563ce-857f-4a9e-839c-83794518bf77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:28.018700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:28.019855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:28.020632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 94/186] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:28.025452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535563ce-857f-4a9e-839c-83794518bf77 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.117684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d97a8e-ceef-4cc8-8b6d-77fff338449f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:28.133663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535563ce-857f-4a9e-839c-83794518bf77 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:28.151491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:28.158178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383113c6-6e5b-43a9-8d48-4901b856e782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:28.159052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383113c6-6e5b-43a9-8d48-4901b856e782 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.159380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383113c6-6e5b-43a9-8d48-4901b856e782 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.159790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383113c6-6e5b-43a9-8d48-4901b856e782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.160635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383113c6-6e5b-43a9-8d48-4901b856e782 None admin] RBAC: Adding query filter params (role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=000e1f52e4464a07aca1f72426e11112, user.id=6e889b4f5d384fef831673b867e27bfa) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:28.160957 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383113c6-6e5b-43a9-8d48-4901b856e782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:28.162516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383113c6-6e5b-43a9-8d48-4901b856e782 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:28.167746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c12b87-338e-4193-b094-75728879ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:28.168629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c12b87-338e-4193-b094-75728879ca65 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.169176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c12b87-338e-4193-b094-75728879ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.169279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 93/187] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:28 2022] GET /identity/v3/role_assignments?user.id=6e889b4f5d384fef831673b867e27bfa&scope.project.id=000e1f52e4464a07aca1f72426e11112&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:28.170266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383113c6-6e5b-43a9-8d48-4901b856e782 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.172218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c12b87-338e-4193-b094-75728879ca65 None admin] PATH_INFO: `/v3/users/3e4928183b684368857a5bcb348fc496` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.192175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c12b87-338e-4193-b094-75728879ca65 None admin] RBAC: Authorizing `identity:get_user(user_id=3e4928183b684368857a5bcb348fc496)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:28.194865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c12b87-338e-4193-b094-75728879ca65 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:28.198086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 95/188] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:28 2022] GET /identity/v3/users/3e4928183b684368857a5bcb348fc496 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:28.199590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c12b87-338e-4193-b094-75728879ca65 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.236461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:28.315715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383113c6-6e5b-43a9-8d48-4901b856e782 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:28.321819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c12b87-338e-4193-b094-75728879ca65 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:28.354860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299edf9-22e0-4ccf-ac4b-7f8fe9f8d5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:28.355711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299edf9-22e0-4ccf-ac4b-7f8fe9f8d5bd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.355998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299edf9-22e0-4ccf-ac4b-7f8fe9f8d5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.356275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299edf9-22e0-4ccf-ac4b-7f8fe9f8d5bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.358230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299edf9-22e0-4ccf-ac4b-7f8fe9f8d5bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:28.359890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299edf9-22e0-4ccf-ac4b-7f8fe9f8d5bd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:28.361716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 94/189] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:28.362336 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299edf9-22e0-4ccf-ac4b-7f8fe9f8d5bd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.365911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fa1ab-5cbd-4461-a837-79fae8e1f2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:28.366884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fa1ab-5cbd-4461-a837-79fae8e1f2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.367506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fa1ab-5cbd-4461-a837-79fae8e1f2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.367783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fa1ab-5cbd-4461-a837-79fae8e1f2b4 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.369046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fa1ab-5cbd-4461-a837-79fae8e1f2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:28.370505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fa1ab-5cbd-4461-a837-79fae8e1f2b4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:28.372504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 96/190] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:28 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:28.376034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fa1ab-5cbd-4461-a837-79fae8e1f2b4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.431751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:28.469125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299edf9-22e0-4ccf-ac4b-7f8fe9f8d5bd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:28.479841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:28.486879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fa1ab-5cbd-4461-a837-79fae8e1f2b4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:28.503065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807d73-c08f-42b3-b8d7-d24fe2de3878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:28.503776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807d73-c08f-42b3-b8d7-d24fe2de3878 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.504066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807d73-c08f-42b3-b8d7-d24fe2de3878 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.504349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807d73-c08f-42b3-b8d7-d24fe2de3878 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.505104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1807d73-c08f-42b3-b8d7-d24fe2de3878 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:28.505417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1807d73-c08f-42b3-b8d7-d24fe2de3878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:28.506807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1807d73-c08f-42b3-b8d7-d24fe2de3878 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:28.515425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 95/191] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:28.516018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683128a-0a74-4896-b7d7-7932a3a33e5d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.516313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683128a-0a74-4896-b7d7-7932a3a33e5d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.516594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683128a-0a74-4896-b7d7-7932a3a33e5d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.517860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 96/192] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:28.518546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683128a-0a74-4896-b7d7-7932a3a33e5d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.526742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e23f0-9009-4dad-a29d-92399e69d933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:28.527425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e23f0-9009-4dad-a29d-92399e69d933 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.527712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e23f0-9009-4dad-a29d-92399e69d933 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.527989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e23f0-9009-4dad-a29d-92399e69d933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.600597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e23f0-9009-4dad-a29d-92399e69d933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:28.602075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e23f0-9009-4dad-a29d-92399e69d933 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:28.625016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683128a-0a74-4896-b7d7-7932a3a33e5d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:28.651114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:28.661539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4026a-5965-4b89-90a2-e24527430bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:28.662217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4026a-5965-4b89-90a2-e24527430bc3 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.662508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4026a-5965-4b89-90a2-e24527430bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.662793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4026a-5965-4b89-90a2-e24527430bc3 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.663534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4026a-5965-4b89-90a2-e24527430bc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:28.664994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4026a-5965-4b89-90a2-e24527430bc3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:28.684246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 97/193] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:00:28 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:28.684901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4026a-5965-4b89-90a2-e24527430bc3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.699657 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:28.708858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 97/194] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:28.709530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e23f0-9009-4dad-a29d-92399e69d933 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.816896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4026a-5965-4b89-90a2-e24527430bc3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:28.829448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e23f0-9009-4dad-a29d-92399e69d933 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:28.856538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fb5ef-c2c9-4c8d-80da-2ea296e414ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:28.857723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fb5ef-c2c9-4c8d-80da-2ea296e414ee None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.857723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fb5ef-c2c9-4c8d-80da-2ea296e414ee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.857723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fb5ef-c2c9-4c8d-80da-2ea296e414ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.858838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fb5ef-c2c9-4c8d-80da-2ea296e414ee None admin] RBAC: Adding query filter params (role.id=2e875b35d6744573ae0287b954108209, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701, user.id=6e889b4f5d384fef831673b867e27bfa) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:28.859043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fb5ef-c2c9-4c8d-80da-2ea296e414ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:28.860656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fb5ef-c2c9-4c8d-80da-2ea296e414ee None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:28.870654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148fb6f-64a2-48f9-aa6a-685d7f264113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:28.870993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 98/195] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:28 2022] GET /identity/v3/role_assignments?user.id=6e889b4f5d384fef831673b867e27bfa&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=2e875b35d6744573ae0287b954108209 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:28.871797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148fb6f-64a2-48f9-aa6a-685d7f264113 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.872154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148fb6f-64a2-48f9-aa6a-685d7f264113 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.872312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379843c6-6922-4f54-bb73-9e16a395964b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:28.872573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148fb6f-64a2-48f9-aa6a-685d7f264113 None admin] PATH_INFO: `/v3/users/3e4928183b684368857a5bcb348fc496` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.872714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379843c6-6922-4f54-bb73-9e16a395964b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:28.872901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379843c6-6922-4f54-bb73-9e16a395964b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:28.873861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148fb6f-64a2-48f9-aa6a-685d7f264113 None admin] RBAC: Authorizing `identity:get_user(user_id=3e4928183b684368857a5bcb348fc496)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:28.875453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148fb6f-64a2-48f9-aa6a-685d7f264113 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:28.877320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 98/196] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:28 2022] GET /identity/v3/users/3e4928183b684368857a5bcb348fc496 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:28.878648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148fb6f-64a2-48f9-aa6a-685d7f264113 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:28.894820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-379843c6-6922-4f54-bb73-9e16a395964b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:28.896161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-379843c6-6922-4f54-bb73-9e16a395964b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:28.959442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:28.966488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:29.005037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 99/197] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:29.006045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379843c6-6922-4f54-bb73-9e16a395964b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:29.016986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148fb6f-64a2-48f9-aa6a-685d7f264113 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:29.061780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f526b4-b063-439f-b8fe-8d1a122d5344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:29.063127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f526b4-b063-439f-b8fe-8d1a122d5344 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:29.063569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f526b4-b063-439f-b8fe-8d1a122d5344 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:29.063979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f526b4-b063-439f-b8fe-8d1a122d5344 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:29.065431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f526b4-b063-439f-b8fe-8d1a122d5344 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:29.067373 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f526b4-b063-439f-b8fe-8d1a122d5344 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:29.069590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 99/198] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:28 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:29.070845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f526b4-b063-439f-b8fe-8d1a122d5344 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:29.141177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379843c6-6922-4f54-bb73-9e16a395964b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:29.194219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1b8ce-924c-481e-8862-ea203806e173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:29.195163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1b8ce-924c-481e-8862-ea203806e173 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:29.195492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1b8ce-924c-481e-8862-ea203806e173 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:29.195780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1b8ce-924c-481e-8862-ea203806e173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:29.202077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f526b4-b063-439f-b8fe-8d1a122d5344 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:29.241683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:29.250781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e858df-ef8d-4144-9b84-dec37f1c244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:29.252034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e858df-ef8d-4144-9b84-dec37f1c244e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:29.252595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e858df-ef8d-4144-9b84-dec37f1c244e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:29.253066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e858df-ef8d-4144-9b84-dec37f1c244e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:29.262575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:29.272600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e858df-ef8d-4144-9b84-dec37f1c244e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c9e858df-ef8d-4144-9b84-dec37f1c244e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:29.281038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:29.281931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:29.282736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:29.285340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 100/199] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:29.286369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e858df-ef8d-4144-9b84-dec37f1c244e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:29.293474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1b8ce-924c-481e-8862-ea203806e173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:29.295486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1b8ce-924c-481e-8862-ea203806e173 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:29.413798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e858df-ef8d-4144-9b84-dec37f1c244e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:29.417019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 100/200] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:29.423091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1b8ce-924c-481e-8862-ea203806e173 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:29.452536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11085cf-3149-4c77-b024-4e14c4fe12f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:29.453209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11085cf-3149-4c77-b024-4e14c4fe12f9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:29.453497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11085cf-3149-4c77-b024-4e14c4fe12f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:29.453783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11085cf-3149-4c77-b024-4e14c4fe12f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:29.454640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11085cf-3149-4c77-b024-4e14c4fe12f9 None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=14787ff2bade4b019afcfdb821d8394b, user.id=3e4928183b684368857a5bcb348fc496) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:29.454961 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11085cf-3149-4c77-b024-4e14c4fe12f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:29.457202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11085cf-3149-4c77-b024-4e14c4fe12f9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:29.463455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 101/201] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:29 2022] GET /identity/v3/role_assignments?user.id=3e4928183b684368857a5bcb348fc496&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:29.472449 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11085cf-3149-4c77-b024-4e14c4fe12f9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:29.519766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:29.560060 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1b8ce-924c-481e-8862-ea203806e173 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:29.615210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:29.634837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304516d-4283-43c8-bba5-371fbfc6df3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:29.635551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304516d-4283-43c8-bba5-371fbfc6df3f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:29.635945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304516d-4283-43c8-bba5-371fbfc6df3f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:29.636117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304516d-4283-43c8-bba5-371fbfc6df3f None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:29.636963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304516d-4283-43c8-bba5-371fbfc6df3f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:29.637279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304516d-4283-43c8-bba5-371fbfc6df3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:29.638779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304516d-4283-43c8-bba5-371fbfc6df3f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:29.651012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 101/202] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:29.656445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304516d-4283-43c8-bba5-371fbfc6df3f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:29.676747 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11085cf-3149-4c77-b024-4e14c4fe12f9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:29.712200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ccfba-7d42-4615-b0fd-78bc8e84a740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:29.713476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ccfba-7d42-4615-b0fd-78bc8e84a740 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:29.713964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ccfba-7d42-4615-b0fd-78bc8e84a740 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:29.714440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ccfba-7d42-4615-b0fd-78bc8e84a740 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:29.715892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ccfba-7d42-4615-b0fd-78bc8e84a740 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:29.718103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ccfba-7d42-4615-b0fd-78bc8e84a740 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:29.720295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 102/203] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:29 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:29.727899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ccfba-7d42-4615-b0fd-78bc8e84a740 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:29.872727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304516d-4283-43c8-bba5-371fbfc6df3f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:29.908711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ccfba-7d42-4615-b0fd-78bc8e84a740 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:29.943845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd964cb-99ab-49cb-88c3-29be1cc8fb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:29.944915 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd964cb-99ab-49cb-88c3-29be1cc8fb60 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:29.945402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd964cb-99ab-49cb-88c3-29be1cc8fb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:29.945914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd964cb-99ab-49cb-88c3-29be1cc8fb60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:29.946955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd964cb-99ab-49cb-88c3-29be1cc8fb60 None admin] RBAC: Adding query filter params (role.id=2e875b35d6744573ae0287b954108209, scope.project.id=14787ff2bade4b019afcfdb821d8394b, user.id=3e4928183b684368857a5bcb348fc496) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:29.947507 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd964cb-99ab-49cb-88c3-29be1cc8fb60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:29.949210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd964cb-99ab-49cb-88c3-29be1cc8fb60 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:29.951582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:29.956316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 103/204] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:29 2022] GET /identity/v3/role_assignments?user.id=3e4928183b684368857a5bcb348fc496&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=2e875b35d6744573ae0287b954108209 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:29.962738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ed6b1-d9ba-4748-aae3-19764e8fae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:29.964097 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ed6b1-d9ba-4748-aae3-19764e8fae6c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:29.964624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ed6b1-d9ba-4748-aae3-19764e8fae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:29.965107 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ed6b1-d9ba-4748-aae3-19764e8fae6c None admin] PATH_INFO: `/v3/users/06f60bd270b04444b3c9d9fe277097bc` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:29.967236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ed6b1-d9ba-4748-aae3-19764e8fae6c None admin] RBAC: Authorizing `identity:get_user(user_id=06f60bd270b04444b3c9d9fe277097bc)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:29.969242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ed6b1-d9ba-4748-aae3-19764e8fae6c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:29.971390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 102/205] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:29 2022] GET /identity/v3/users/06f60bd270b04444b3c9d9fe277097bc => generated 289 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:29.980453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd964cb-99ab-49cb-88c3-29be1cc8fb60 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:30.130077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd964cb-99ab-49cb-88c3-29be1cc8fb60 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:30.174842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa802ea-c008-45fd-9f37-63729fd57476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:30.175539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa802ea-c008-45fd-9f37-63729fd57476 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:30.175830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa802ea-c008-45fd-9f37-63729fd57476 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:30.176113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa802ea-c008-45fd-9f37-63729fd57476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:30.261304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:30.277227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa802ea-c008-45fd-9f37-63729fd57476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:30.278734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa802ea-c008-45fd-9f37-63729fd57476 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:30.402321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 104/206] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:30.408425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ed6b1-d9ba-4748-aae3-19764e8fae6c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:30.420372 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:30.541882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ed6b1-d9ba-4748-aae3-19764e8fae6c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:30.596090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324a940-1803-4a78-a2de-cefccc90b804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:30.597033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324a940-1803-4a78-a2de-cefccc90b804 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:30.597322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324a940-1803-4a78-a2de-cefccc90b804 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:30.597602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324a940-1803-4a78-a2de-cefccc90b804 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:30.598847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324a940-1803-4a78-a2de-cefccc90b804 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:30.600356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324a940-1803-4a78-a2de-cefccc90b804 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:30.602735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 103/207] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:30 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:30.614030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa802ea-c008-45fd-9f37-63729fd57476 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:30.730073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:30.747488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa802ea-c008-45fd-9f37-63729fd57476 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:30.799677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77ae82-3bd3-4513-b5df-501b3c9b9f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:30.800265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77ae82-3bd3-4513-b5df-501b3c9b9f22 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:30.800552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77ae82-3bd3-4513-b5df-501b3c9b9f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:30.800833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77ae82-3bd3-4513-b5df-501b3c9b9f22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:30.801678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77ae82-3bd3-4513-b5df-501b3c9b9f22 None admin] RBAC: Adding query filter params (role.id=a5a3836660244b6eb47bc34d9d97907a, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701, user.id=06f60bd270b04444b3c9d9fe277097bc) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:30.801993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77ae82-3bd3-4513-b5df-501b3c9b9f22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:30.803654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77ae82-3bd3-4513-b5df-501b3c9b9f22 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:30.812608 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 105/208] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:30 2022] GET /identity/v3/role_assignments?user.id=06f60bd270b04444b3c9d9fe277097bc&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=a5a3836660244b6eb47bc34d9d97907a => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:31.267235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5387d5-a237-443d-808b-16f59dc0c8dd None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.267614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5387d5-a237-443d-808b-16f59dc0c8dd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.267794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5387d5-a237-443d-808b-16f59dc0c8dd None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.269285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 104/209] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:31.274368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb043d-bc9d-4984-9fd1-45e769ff19f9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.274813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb043d-bc9d-4984-9fd1-45e769ff19f9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.275000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb043d-bc9d-4984-9fd1-45e769ff19f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.293531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b5eb043d-bc9d-4984-9fd1-45e769ff19f9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:31.295380 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b5eb043d-bc9d-4984-9fd1-45e769ff19f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:31.368597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1df1b9-8dce-4281-a297-33db39e886f4 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.368908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1df1b9-8dce-4281-a297-33db39e886f4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.369163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1df1b9-8dce-4281-a297-33db39e886f4 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.370685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 105/210] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:31.374645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 106/211] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:31.377570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a9cd0-ccde-46e8-983c-378a079a6c0f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.378133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a9cd0-ccde-46e8-983c-378a079a6c0f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.378628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a9cd0-ccde-46e8-983c-378a079a6c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.399556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb043d-bc9d-4984-9fd1-45e769ff19f9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:31.404834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1e1a9cd0-ccde-46e8-983c-378a079a6c0f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:31.404971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1e1a9cd0-ccde-46e8-983c-378a079a6c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:31.476228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 106/212] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:31.500024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:31.503826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a9cd0-ccde-46e8-983c-378a079a6c0f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:31.507067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:31.528571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb043d-bc9d-4984-9fd1-45e769ff19f9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:31.575445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf831f11-0662-4dc2-9cb6-c7aaf9f197f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:31.577215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf831f11-0662-4dc2-9cb6-c7aaf9f197f4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.577745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf831f11-0662-4dc2-9cb6-c7aaf9f197f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.578282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf831f11-0662-4dc2-9cb6-c7aaf9f197f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.590423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf831f11-0662-4dc2-9cb6-c7aaf9f197f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-cf831f11-0662-4dc2-9cb6-c7aaf9f197f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:31.602461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:31.603602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:31.604617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:31.608696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 107/213] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:31.609781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf831f11-0662-4dc2-9cb6-c7aaf9f197f4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:31.632914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a9cd0-ccde-46e8-983c-378a079a6c0f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:31.673662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67e180-d22c-45bb-ada5-2ba5fd2468fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:31.674482 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67e180-d22c-45bb-ada5-2ba5fd2468fa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.674905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67e180-d22c-45bb-ada5-2ba5fd2468fa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.675355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67e180-d22c-45bb-ada5-2ba5fd2468fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.676802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67e180-d22c-45bb-ada5-2ba5fd2468fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:31.678953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67e180-d22c-45bb-ada5-2ba5fd2468fa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:31.681294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 107/214] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:31.686619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67e180-d22c-45bb-ada5-2ba5fd2468fa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:31.725864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf831f11-0662-4dc2-9cb6-c7aaf9f197f4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:31.760603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66014191-86c4-47b9-96f2-435c326374eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:31.761720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66014191-86c4-47b9-96f2-435c326374eb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.762201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66014191-86c4-47b9-96f2-435c326374eb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.762669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66014191-86c4-47b9-96f2-435c326374eb None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.762880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:31.764533 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66014191-86c4-47b9-96f2-435c326374eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:31.765077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66014191-86c4-47b9-96f2-435c326374eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:31.766710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66014191-86c4-47b9-96f2-435c326374eb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:31.775718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 108/215] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:31.782135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66014191-86c4-47b9-96f2-435c326374eb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:31.785854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:31.799574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67e180-d22c-45bb-ada5-2ba5fd2468fa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:31.835988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaf74d-2292-4605-826c-3325895da369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:31.836764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaf74d-2292-4605-826c-3325895da369 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.837059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaf74d-2292-4605-826c-3325895da369 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.837341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaf74d-2292-4605-826c-3325895da369 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.838059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaf74d-2292-4605-826c-3325895da369 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:31.839535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaf74d-2292-4605-826c-3325895da369 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:31.859665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 108/216] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:00:31 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:00:31.865982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6313926-3392-40a1-968d-abced7a42254 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.866375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6313926-3392-40a1-968d-abced7a42254 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.866772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6313926-3392-40a1-968d-abced7a42254 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.868641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 109/217] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:31.874422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66699d-ccbc-45cb-b309-062cbef5650b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.874775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66699d-ccbc-45cb-b309-062cbef5650b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.875004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66699d-ccbc-45cb-b309-062cbef5650b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.894310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7e66699d-ccbc-45cb-b309-062cbef5650b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:31.895527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7e66699d-ccbc-45cb-b309-062cbef5650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:31.919589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66014191-86c4-47b9-96f2-435c326374eb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:31.968290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19b361-c24e-4ac5-ab2d-a4a3887decc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:31.969207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19b361-c24e-4ac5-ab2d-a4a3887decc5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:31.969499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19b361-c24e-4ac5-ab2d-a4a3887decc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:31.969781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19b361-c24e-4ac5-ab2d-a4a3887decc5 None admin] PATH_INFO: `/v3/users/f0a5288aaa1e468eb46a793a8188bebf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:31.977135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 110/218] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:31.994818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19b361-c24e-4ac5-ab2d-a4a3887decc5 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a5288aaa1e468eb46a793a8188bebf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:31.996446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19b361-c24e-4ac5-ab2d-a4a3887decc5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:31.999132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 109/219] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:31 2022] GET /identity/v3/users/f0a5288aaa1e468eb46a793a8188bebf => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:32.003778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66699d-ccbc-45cb-b309-062cbef5650b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:32.004564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19b361-c24e-4ac5-ab2d-a4a3887decc5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:32.074704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:32.086775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:32.140417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19b361-c24e-4ac5-ab2d-a4a3887decc5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:32.153532 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66699d-ccbc-45cb-b309-062cbef5650b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:32.187474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fba0a-24e9-4db2-9fe4-6e0eb66af4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:32.188011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fba0a-24e9-4db2-9fe4-6e0eb66af4ad None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:32.188294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fba0a-24e9-4db2-9fe4-6e0eb66af4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:32.188578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fba0a-24e9-4db2-9fe4-6e0eb66af4ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:32.201483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c9f2-7e21-4514-aa51-785fb9a622e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:32.202298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c9f2-7e21-4514-aa51-785fb9a622e7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:32.202591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c9f2-7e21-4514-aa51-785fb9a622e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:32.202871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c9f2-7e21-4514-aa51-785fb9a622e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:32.218636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399c9f2-7e21-4514-aa51-785fb9a622e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a399c9f2-7e21-4514-aa51-785fb9a622e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:32.228638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:32.229734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:32.230801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 111/220] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:32.239638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c9f2-7e21-4514-aa51-785fb9a622e7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:32.282780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fba0a-24e9-4db2-9fe4-6e0eb66af4ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:32.284421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fba0a-24e9-4db2-9fe4-6e0eb66af4ad None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:32.365754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:32.376114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c9f2-7e21-4514-aa51-785fb9a622e7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:32.397353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:32.418408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc0a51-91ed-4421-beaa-11482a0dd858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:32.419536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc0a51-91ed-4421-beaa-11482a0dd858 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:32.420024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc0a51-91ed-4421-beaa-11482a0dd858 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:32.420486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc0a51-91ed-4421-beaa-11482a0dd858 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:32.421489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc0a51-91ed-4421-beaa-11482a0dd858 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:32.422001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc0a51-91ed-4421-beaa-11482a0dd858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:32.423195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 110/221] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 22 11:00:32.423916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc0a51-91ed-4421-beaa-11482a0dd858 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:32.428810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fba0a-24e9-4db2-9fe4-6e0eb66af4ad None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:32.435776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 112/222] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:32.441269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc0a51-91ed-4421-beaa-11482a0dd858 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:32.533149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fba0a-24e9-4db2-9fe4-6e0eb66af4ad None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:32.553331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc0a51-91ed-4421-beaa-11482a0dd858 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:32.568956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30accb-0c32-4a14-9a78-8ea2289838a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:32.569835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30accb-0c32-4a14-9a78-8ea2289838a1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:32.570127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30accb-0c32-4a14-9a78-8ea2289838a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:32.570409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30accb-0c32-4a14-9a78-8ea2289838a1 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:32.571703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30accb-0c32-4a14-9a78-8ea2289838a1 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:32.573128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30accb-0c32-4a14-9a78-8ea2289838a1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:32.574873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 111/223] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:32 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:32.581307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30accb-0c32-4a14-9a78-8ea2289838a1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:32.603147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d34b4-0d10-40b1-b44d-260e776891ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:32.604460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d34b4-0d10-40b1-b44d-260e776891ad None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:32.604944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d34b4-0d10-40b1-b44d-260e776891ad None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:32.605474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d34b4-0d10-40b1-b44d-260e776891ad None admin] PATH_INFO: `/v3/users/3e4928183b684368857a5bcb348fc496` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:32.607071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d34b4-0d10-40b1-b44d-260e776891ad None admin] RBAC: Authorizing `identity:get_user(user_id=3e4928183b684368857a5bcb348fc496)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:32.608828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d34b4-0d10-40b1-b44d-260e776891ad None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:32.611224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 113/224] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:32 2022] GET /identity/v3/users/3e4928183b684368857a5bcb348fc496 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:32.613334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:32.619109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d34b4-0d10-40b1-b44d-260e776891ad None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:32.674486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:32.690201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30accb-0c32-4a14-9a78-8ea2289838a1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:32.726494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b35de-4db4-4c26-963a-ec22e74f2b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:32.727163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b35de-4db4-4c26-963a-ec22e74f2b11 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:32.727497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b35de-4db4-4c26-963a-ec22e74f2b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:32.727779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b35de-4db4-4c26-963a-ec22e74f2b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:32.728627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b35de-4db4-4c26-963a-ec22e74f2b11 None admin] RBAC: Adding query filter params (role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=14787ff2bade4b019afcfdb821d8394b, user.id=f0a5288aaa1e468eb46a793a8188bebf) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:32.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b35de-4db4-4c26-963a-ec22e74f2b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:32.730421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b35de-4db4-4c26-963a-ec22e74f2b11 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:32.735915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d34b4-0d10-40b1-b44d-260e776891ad None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:32.736384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 112/225] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:32 2022] GET /identity/v3/role_assignments?user.id=f0a5288aaa1e468eb46a793a8188bebf&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:32.737767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def602f-d072-411f-844f-e64496169060 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:32.743560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def602f-d072-411f-844f-e64496169060 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:32.743957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def602f-d072-411f-844f-e64496169060 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:32.745472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 113/226] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:32.751811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75fd2-4fb0-44eb-9102-7eb595b6265d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:32.752105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75fd2-4fb0-44eb-9102-7eb595b6265d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:32.752391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75fd2-4fb0-44eb-9102-7eb595b6265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:32.780996 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469ad2f-7418-444a-b326-fa68a01226fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:32.782356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ad2f-7418-444a-b326-fa68a01226fa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:32.782846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ad2f-7418-444a-b326-fa68a01226fa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:32.783352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ad2f-7418-444a-b326-fa68a01226fa None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:32.785110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469ad2f-7418-444a-b326-fa68a01226fa None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:32.786023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-95f75fd2-4fb0-44eb-9102-7eb595b6265d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:32.787062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469ad2f-7418-444a-b326-fa68a01226fa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:32.787278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-95f75fd2-4fb0-44eb-9102-7eb595b6265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:32.789413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 114/227] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:32 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:32.796717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469ad2f-7418-444a-b326-fa68a01226fa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:32.864167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 114/228] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:32.887885 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75fd2-4fb0-44eb-9102-7eb595b6265d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:32.918307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:33.005119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75fd2-4fb0-44eb-9102-7eb595b6265d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:33.009229 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469ad2f-7418-444a-b326-fa68a01226fa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:33.031618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:33.041557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4ba4d-36bf-4a9d-9749-69057a18d108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:33.042430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4ba4d-36bf-4a9d-9749-69057a18d108 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:33.042723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4ba4d-36bf-4a9d-9749-69057a18d108 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:33.043004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4ba4d-36bf-4a9d-9749-69057a18d108 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:33.054887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4ba4d-36bf-4a9d-9749-69057a18d108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:33.056621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5b0f7-2c7e-4b6e-894a-65319d1f35e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:33.057791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5b0f7-2c7e-4b6e-894a-65319d1f35e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:33.058273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5b0f7-2c7e-4b6e-894a-65319d1f35e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:33.058868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5b0f7-2c7e-4b6e-894a-65319d1f35e5 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b/users/3e4928183b684368857a5bcb348fc496/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:33.061729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5b0f7-2c7e-4b6e-894a-65319d1f35e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=14787ff2bade4b019afcfdb821d8394b, user_id=3e4928183b684368857a5bcb348fc496, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:33.063452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5b0f7-2c7e-4b6e-894a-65319d1f35e5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-31c4ba4d-36bf-4a9d-9749-69057a18d108 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:33.064169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:33.065053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:33.065977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 115/229] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:33.072258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4ba4d-36bf-4a9d-9749-69057a18d108 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:33.086672 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 115/230] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:00:32 2022] PUT /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b/users/3e4928183b684368857a5bcb348fc496/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:33.173152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:33.199624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4ba4d-36bf-4a9d-9749-69057a18d108 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:33.235739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24f848-1e39-4831-b8d6-820f2524679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:33.236690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24f848-1e39-4831-b8d6-820f2524679c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:33.237120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24f848-1e39-4831-b8d6-820f2524679c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:33.237537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24f848-1e39-4831-b8d6-820f2524679c None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:33.238431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24f848-1e39-4831-b8d6-820f2524679c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:33.238879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24f848-1e39-4831-b8d6-820f2524679c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:33.240752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24f848-1e39-4831-b8d6-820f2524679c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:33.250755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 116/231] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:33.260430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5b0f7-2c7e-4b6e-894a-65319d1f35e5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:33.394426 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5b0f7-2c7e-4b6e-894a-65319d1f35e5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:33.442591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3972043-13e6-4a75-b80e-5bad488ba60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:33.443740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3972043-13e6-4a75-b80e-5bad488ba60c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:33.444170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3972043-13e6-4a75-b80e-5bad488ba60c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:33.444590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3972043-13e6-4a75-b80e-5bad488ba60c None admin] PATH_INFO: `/v3/users/3e4928183b684368857a5bcb348fc496` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:33.445945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3972043-13e6-4a75-b80e-5bad488ba60c None admin] RBAC: Authorizing `identity:get_user(user_id=3e4928183b684368857a5bcb348fc496)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:33.447643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3972043-13e6-4a75-b80e-5bad488ba60c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:33.449517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 116/232] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:33 2022] GET /identity/v3/users/3e4928183b684368857a5bcb348fc496 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:33.455215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24f848-1e39-4831-b8d6-820f2524679c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:33.583400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24f848-1e39-4831-b8d6-820f2524679c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:33.621631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337c55b-0eab-4dcc-9e9b-848874280e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:33.622586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337c55b-0eab-4dcc-9e9b-848874280e4c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:33.622880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337c55b-0eab-4dcc-9e9b-848874280e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:33.623166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337c55b-0eab-4dcc-9e9b-848874280e4c None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:33.624619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337c55b-0eab-4dcc-9e9b-848874280e4c None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:33.626087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337c55b-0eab-4dcc-9e9b-848874280e4c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:33.627880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 117/233] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:33 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:33.634167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3972043-13e6-4a75-b80e-5bad488ba60c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:33.706219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:33.780841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3972043-13e6-4a75-b80e-5bad488ba60c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:33.825047 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145dbb0-567d-4e48-83a7-c8059fa7e8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:33.826469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145dbb0-567d-4e48-83a7-c8059fa7e8ca None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:33.826754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145dbb0-567d-4e48-83a7-c8059fa7e8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:33.827035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145dbb0-567d-4e48-83a7-c8059fa7e8ca None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b/users/3e4928183b684368857a5bcb348fc496/roles/2e875b35d6744573ae0287b954108209` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:33.833131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145dbb0-567d-4e48-83a7-c8059fa7e8ca None admin] RBAC: Authorizing `identity:create_grant(project_id=14787ff2bade4b019afcfdb821d8394b, user_id=3e4928183b684368857a5bcb348fc496, role_id=2e875b35d6744573ae0287b954108209)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:33.834608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145dbb0-567d-4e48-83a7-c8059fa7e8ca None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:33.865569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 117/234] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:00:33 2022] PUT /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b/users/3e4928183b684368857a5bcb348fc496/roles/2e875b35d6744573ae0287b954108209 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:00:34.327151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328057b4-0fe8-44ea-a443-9b3866ed1439 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:34.327875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328057b4-0fe8-44ea-a443-9b3866ed1439 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:34.328379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328057b4-0fe8-44ea-a443-9b3866ed1439 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:34.329973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 118/235] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:34.335695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fbf84-16e6-4d62-8ff0-ae61b91e37e8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:34.336226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fbf84-16e6-4d62-8ff0-ae61b91e37e8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:34.336699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fbf84-16e6-4d62-8ff0-ae61b91e37e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:34.358031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-098fbf84-16e6-4d62-8ff0-ae61b91e37e8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:34.359745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-098fbf84-16e6-4d62-8ff0-ae61b91e37e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:34.450357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 118/236] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:34.491470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328057b4-0fe8-44ea-a443-9b3866ed1439 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:34.514062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:34.645535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328057b4-0fe8-44ea-a443-9b3866ed1439 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:34.686202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcafcb-b678-4e1e-ae67-c3dca6fabfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:34.687001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcafcb-b678-4e1e-ae67-c3dca6fabfab None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:34.687301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcafcb-b678-4e1e-ae67-c3dca6fabfab None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:34.687586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcafcb-b678-4e1e-ae67-c3dca6fabfab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:34.704127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcafcb-b678-4e1e-ae67-c3dca6fabfab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d2fcafcb-b678-4e1e-ae67-c3dca6fabfab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:34.711792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:34.712652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:34.713430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 119/237] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:34.722429 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fbf84-16e6-4d62-8ff0-ae61b91e37e8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:34.771655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:34.862281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fbf84-16e6-4d62-8ff0-ae61b91e37e8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:34.909116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023881b-ff48-46ee-867e-fba0668134a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:34.909799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023881b-ff48-46ee-867e-fba0668134a5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:34.910084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023881b-ff48-46ee-867e-fba0668134a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:34.910365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023881b-ff48-46ee-867e-fba0668134a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:34.911121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023881b-ff48-46ee-867e-fba0668134a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:34.911455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023881b-ff48-46ee-867e-fba0668134a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:34.912825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023881b-ff48-46ee-867e-fba0668134a5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:34.922703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 119/238] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:34.929077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcafcb-b678-4e1e-ae67-c3dca6fabfab None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:35.028235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:35.059046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcafcb-b678-4e1e-ae67-c3dca6fabfab None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:35.072895 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e687a-c51e-4731-b892-aee33093a9e0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:35.073500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e687a-c51e-4731-b892-aee33093a9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:35.073975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e687a-c51e-4731-b892-aee33093a9e0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:35.076204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 120/239] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:35.103505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867b384-8690-4fa5-a527-631d7c968747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:35.104379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867b384-8690-4fa5-a527-631d7c968747 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:35.104659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867b384-8690-4fa5-a527-631d7c968747 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:35.104933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867b384-8690-4fa5-a527-631d7c968747 None admin] PATH_INFO: `/v3/users/da57395c125b45da988aa4fd5b4d252d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:35.105749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc0612-7cdb-40de-8f50-25706648dd5c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:35.106237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc0612-7cdb-40de-8f50-25706648dd5c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:35.106754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc0612-7cdb-40de-8f50-25706648dd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:35.119769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867b384-8690-4fa5-a527-631d7c968747 None admin] RBAC: Authorizing `identity:get_user(user_id=da57395c125b45da988aa4fd5b4d252d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:35.121190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867b384-8690-4fa5-a527-631d7c968747 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:35.124002 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 120/240] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:34 2022] GET /identity/v3/users/da57395c125b45da988aa4fd5b4d252d => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:35.135363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867b384-8690-4fa5-a527-631d7c968747 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:35.138709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6cdc0612-7cdb-40de-8f50-25706648dd5c None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:35.140462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6cdc0612-7cdb-40de-8f50-25706648dd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:35.233065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867b384-8690-4fa5-a527-631d7c968747 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:35.235034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 121/241] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:35.288337 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc0612-7cdb-40de-8f50-25706648dd5c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:35.292442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:35.295112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028947db-f41f-4d35-b5cc-335baea1f853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:35.295974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028947db-f41f-4d35-b5cc-335baea1f853 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:35.296423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028947db-f41f-4d35-b5cc-335baea1f853 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:35.296816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028947db-f41f-4d35-b5cc-335baea1f853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:35.360570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:35.405015 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028947db-f41f-4d35-b5cc-335baea1f853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:35.406733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028947db-f41f-4d35-b5cc-335baea1f853 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:35.460824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc0612-7cdb-40de-8f50-25706648dd5c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:35.500296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63282bf3-88ed-4c1a-bb65-89f0cf6e93cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:35.503282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282bf3-88ed-4c1a-bb65-89f0cf6e93cc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:35.503282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282bf3-88ed-4c1a-bb65-89f0cf6e93cc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:35.503282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63282bf3-88ed-4c1a-bb65-89f0cf6e93cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:35.503282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63282bf3-88ed-4c1a-bb65-89f0cf6e93cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:35.504863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63282bf3-88ed-4c1a-bb65-89f0cf6e93cc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:35.506813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 122/242] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:35.512158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63282bf3-88ed-4c1a-bb65-89f0cf6e93cc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:35.619335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 121/243] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:35.621267 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf76de-21e1-4983-a791-a2614af5239c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:35.621701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf76de-21e1-4983-a791-a2614af5239c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:35.622347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf76de-21e1-4983-a791-a2614af5239c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:35.624381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 122/244] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:35.625769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf76de-21e1-4983-a791-a2614af5239c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:35.628109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63282bf3-88ed-4c1a-bb65-89f0cf6e93cc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:35.647615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:35.663706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:35.670224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863d99c-b22a-4df2-9c94-7048fa4ec5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:35.671158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863d99c-b22a-4df2-9c94-7048fa4ec5b4 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:35.671665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863d99c-b22a-4df2-9c94-7048fa4ec5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:35.672136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863d99c-b22a-4df2-9c94-7048fa4ec5b4 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:35.673042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863d99c-b22a-4df2-9c94-7048fa4ec5b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:35.674713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863d99c-b22a-4df2-9c94-7048fa4ec5b4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:35.698570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 123/245] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:00:35 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:35.700616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545c5ea-e0a4-4319-9dce-869d059dcafe None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:35.701174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545c5ea-e0a4-4319-9dce-869d059dcafe None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:35.701664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545c5ea-e0a4-4319-9dce-869d059dcafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:35.723624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2545c5ea-e0a4-4319-9dce-869d059dcafe None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:35.726278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2545c5ea-e0a4-4319-9dce-869d059dcafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:35.811924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 124/246] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:35.834768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545c5ea-e0a4-4319-9dce-869d059dcafe None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:35.884062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf76de-21e1-4983-a791-a2614af5239c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:35.943930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:35.949166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367126dc-b2cc-47aa-b942-47f1e0c6c8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:35.950142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367126dc-b2cc-47aa-b942-47f1e0c6c8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:35.951050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367126dc-b2cc-47aa-b942-47f1e0c6c8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:35.952413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367126dc-b2cc-47aa-b942-47f1e0c6c8b3 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:35.953324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367126dc-b2cc-47aa-b942-47f1e0c6c8b3 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:35.954799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367126dc-b2cc-47aa-b942-47f1e0c6c8b3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:35.957787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 123/247] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:35 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:35.966017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367126dc-b2cc-47aa-b942-47f1e0c6c8b3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:35.984279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545c5ea-e0a4-4319-9dce-869d059dcafe None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:36.024744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9fb6b-6ec4-4a92-ae9a-6aa396da0cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:36.025544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9fb6b-6ec4-4a92-ae9a-6aa396da0cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.025826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9fb6b-6ec4-4a92-ae9a-6aa396da0cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.026098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9fb6b-6ec4-4a92-ae9a-6aa396da0cd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.042856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9fb6b-6ec4-4a92-ae9a-6aa396da0cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-02a9fb6b-6ec4-4a92-ae9a-6aa396da0cd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:36.052296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:36.053322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 125/248] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:36.057174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa75b2-2f84-48ac-83a8-a6fd4654b772 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.057968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa75b2-2f84-48ac-83a8-a6fd4654b772 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.057968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa75b2-2f84-48ac-83a8-a6fd4654b772 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.057968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 126/249] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:36.065037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa75b2-2f84-48ac-83a8-a6fd4654b772 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:36.098514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:36.106229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367126dc-b2cc-47aa-b942-47f1e0c6c8b3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:36.160487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d67b7-1b35-4967-b3c7-99d5a4fc03bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:36.161158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d67b7-1b35-4967-b3c7-99d5a4fc03bc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.161451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d67b7-1b35-4967-b3c7-99d5a4fc03bc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.161732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d67b7-1b35-4967-b3c7-99d5a4fc03bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.162582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d67b7-1b35-4967-b3c7-99d5a4fc03bc None admin] RBAC: Adding query filter params (role.id=a5a3836660244b6eb47bc34d9d97907a, scope.project.id=14787ff2bade4b019afcfdb821d8394b, user.id=da57395c125b45da988aa4fd5b4d252d) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:36.162894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d67b7-1b35-4967-b3c7-99d5a4fc03bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:36.164407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d67b7-1b35-4967-b3c7-99d5a4fc03bc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:36.173065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 124/250] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:35 2022] GET /identity/v3/role_assignments?user.id=da57395c125b45da988aa4fd5b4d252d&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=a5a3836660244b6eb47bc34d9d97907a => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:36.174466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39f275-0f03-4747-8636-64f2982149a5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.174733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39f275-0f03-4747-8636-64f2982149a5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.175020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39f275-0f03-4747-8636-64f2982149a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.196145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2f39f275-0f03-4747-8636-64f2982149a5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:36.197295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f39f275-0f03-4747-8636-64f2982149a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:36.204479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa75b2-2f84-48ac-83a8-a6fd4654b772 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:36.255454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554eeb7-82a6-4a84-a6d3-e653d4e0ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:36.256148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554eeb7-82a6-4a84-a6d3-e653d4e0ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.256433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554eeb7-82a6-4a84-a6d3-e653d4e0ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.256719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554eeb7-82a6-4a84-a6d3-e653d4e0ccb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.257490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554eeb7-82a6-4a84-a6d3-e653d4e0ccb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:36.257795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554eeb7-82a6-4a84-a6d3-e653d4e0ccb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:36.259215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554eeb7-82a6-4a84-a6d3-e653d4e0ccb3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:36.269952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 127/251] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:36.275556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554eeb7-82a6-4a84-a6d3-e653d4e0ccb3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:36.279610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:36.279929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 125/252] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:36.304454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39f275-0f03-4747-8636-64f2982149a5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:36.387656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554eeb7-82a6-4a84-a6d3-e653d4e0ccb3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:36.424949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f677d-5d37-4a95-9b30-219200c34624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:36.425854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f677d-5d37-4a95-9b30-219200c34624 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.426135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f677d-5d37-4a95-9b30-219200c34624 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.426412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f677d-5d37-4a95-9b30-219200c34624 None admin] PATH_INFO: `/v3/users/f0a5288aaa1e468eb46a793a8188bebf` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.427737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f677d-5d37-4a95-9b30-219200c34624 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a5288aaa1e468eb46a793a8188bebf)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:36.429243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f677d-5d37-4a95-9b30-219200c34624 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:36.431174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 128/253] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:36 2022] GET /identity/v3/users/f0a5288aaa1e468eb46a793a8188bebf => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:36.436502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8716b-30aa-419b-98e5-1f2fab0a16a4 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.436799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8716b-30aa-419b-98e5-1f2fab0a16a4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.437152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8716b-30aa-419b-98e5-1f2fab0a16a4 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.438719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 129/254] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:36.441699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8716b-30aa-419b-98e5-1f2fab0a16a4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:36.464408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:36.489036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39f275-0f03-4747-8636-64f2982149a5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:36.540759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3777a-2d2d-4875-8f98-c3521acef958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:36.541567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3777a-2d2d-4875-8f98-c3521acef958 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.541855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3777a-2d2d-4875-8f98-c3521acef958 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.542135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3777a-2d2d-4875-8f98-c3521acef958 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.554035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d3777a-2d2d-4875-8f98-c3521acef958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-22d3777a-2d2d-4875-8f98-c3521acef958 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:36.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:36.562784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:36.564034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 126/255] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:36.564894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f8479-31f3-423f-b882-85c453a1c14a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.565120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f8479-31f3-423f-b882-85c453a1c14a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.565284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f8479-31f3-423f-b882-85c453a1c14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.570681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:36.585269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b31f8479-31f3-423f-b882-85c453a1c14a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:36.586479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b31f8479-31f3-423f-b882-85c453a1c14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:36.660286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 127/256] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:36.661197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f8479-31f3-423f-b882-85c453a1c14a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:36.684984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8716b-30aa-419b-98e5-1f2fab0a16a4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:36.751739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:36.772027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f8479-31f3-423f-b882-85c453a1c14a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:36.808480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eaf7f-5cd8-4ec8-af36-c202a788fc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:36.809159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eaf7f-5cd8-4ec8-af36-c202a788fc26 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.809546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eaf7f-5cd8-4ec8-af36-c202a788fc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.809872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eaf7f-5cd8-4ec8-af36-c202a788fc26 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.810633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62eaf7f-5cd8-4ec8-af36-c202a788fc26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:36.810945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62eaf7f-5cd8-4ec8-af36-c202a788fc26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:36.812516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62eaf7f-5cd8-4ec8-af36-c202a788fc26 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:36.813306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41408daa-c54b-493f-b7ed-e57c20dc8571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:36.814738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41408daa-c54b-493f-b7ed-e57c20dc8571 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.815226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41408daa-c54b-493f-b7ed-e57c20dc8571 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.815727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41408daa-c54b-493f-b7ed-e57c20dc8571 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.819994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41408daa-c54b-493f-b7ed-e57c20dc8571 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:36.821335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 128/257] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:36.822010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41408daa-c54b-493f-b7ed-e57c20dc8571 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:36.822186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eaf7f-5cd8-4ec8-af36-c202a788fc26 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:36.824246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 130/258] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:36 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:36.830861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41408daa-c54b-493f-b7ed-e57c20dc8571 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:36.932029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eaf7f-5cd8-4ec8-af36-c202a788fc26 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:36.971288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ea669-8dd1-4e0e-89ba-194c96adf5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:36.971288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ea669-8dd1-4e0e-89ba-194c96adf5bc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:36.971288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ea669-8dd1-4e0e-89ba-194c96adf5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:36.971288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ea669-8dd1-4e0e-89ba-194c96adf5bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:36.980400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ea669-8dd1-4e0e-89ba-194c96adf5bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-674ea669-8dd1-4e0e-89ba-194c96adf5bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:36.990702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:36.991647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:36.992454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:36.994599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 129/259] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:36.995447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ea669-8dd1-4e0e-89ba-194c96adf5bc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:37.005186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:37.110324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41408daa-c54b-493f-b7ed-e57c20dc8571 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:37.111151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ea669-8dd1-4e0e-89ba-194c96adf5bc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:37.163664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976961f3-7730-4580-adc4-8b28161f5a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:37.164964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976961f3-7730-4580-adc4-8b28161f5a12 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:37.165565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976961f3-7730-4580-adc4-8b28161f5a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:37.165851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976961f3-7730-4580-adc4-8b28161f5a12 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b/users/f0a5288aaa1e468eb46a793a8188bebf/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:37.167946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976961f3-7730-4580-adc4-8b28161f5a12 None admin] RBAC: Authorizing `identity:create_grant(project_id=14787ff2bade4b019afcfdb821d8394b, user_id=f0a5288aaa1e468eb46a793a8188bebf, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:37.169929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976961f3-7730-4580-adc4-8b28161f5a12 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:37.177505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9d3b6-1698-4017-be10-8bbfa7603800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:37.178536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d3b6-1698-4017-be10-8bbfa7603800 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:37.178833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d3b6-1698-4017-be10-8bbfa7603800 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:37.179135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d3b6-1698-4017-be10-8bbfa7603800 None admin] PATH_INFO: `/v3/users/3e4928183b684368857a5bcb348fc496` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:37.184013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9d3b6-1698-4017-be10-8bbfa7603800 None admin] RBAC: Authorizing `identity:get_user(user_id=3e4928183b684368857a5bcb348fc496)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:37.186050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9d3b6-1698-4017-be10-8bbfa7603800 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:37.187953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 131/260] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:36 2022] GET /identity/v3/users/3e4928183b684368857a5bcb348fc496 => generated 283 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:37.188504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9d3b6-1698-4017-be10-8bbfa7603800 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:37.188672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 130/261] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:00:36 2022] PUT /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b/users/f0a5288aaa1e468eb46a793a8188bebf/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:37.194756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976961f3-7730-4580-adc4-8b28161f5a12 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:37.206754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:37.285641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:37.342374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976961f3-7730-4580-adc4-8b28161f5a12 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:37.388909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86125dab-6530-4602-ad47-6195946d8b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:37.389717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86125dab-6530-4602-ad47-6195946d8b84 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:37.390011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86125dab-6530-4602-ad47-6195946d8b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:37.390288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86125dab-6530-4602-ad47-6195946d8b84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:37.448805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9d3b6-1698-4017-be10-8bbfa7603800 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:37.483720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86125dab-6530-4602-ad47-6195946d8b84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:37.485406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86125dab-6530-4602-ad47-6195946d8b84 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:37.498061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45db8d-688c-41eb-a42b-bd7fe1c01ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:37.498803 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45db8d-688c-41eb-a42b-bd7fe1c01ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:37.499096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45db8d-688c-41eb-a42b-bd7fe1c01ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:37.499411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45db8d-688c-41eb-a42b-bd7fe1c01ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:37.500188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45db8d-688c-41eb-a42b-bd7fe1c01ae9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:37.500511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45db8d-688c-41eb-a42b-bd7fe1c01ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:37.502711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45db8d-688c-41eb-a42b-bd7fe1c01ae9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:37.513872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 132/262] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:37.524060 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45db8d-688c-41eb-a42b-bd7fe1c01ae9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:37.598633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:37.635637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:37.698433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45db8d-688c-41eb-a42b-bd7fe1c01ae9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:37.715166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 131/263] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:37.722083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86125dab-6530-4602-ad47-6195946d8b84 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:37.756074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523b9c1-1f78-4748-8ef6-5f2f7325dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:37.757323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523b9c1-1f78-4748-8ef6-5f2f7325dc34 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:37.757838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523b9c1-1f78-4748-8ef6-5f2f7325dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:37.758349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523b9c1-1f78-4748-8ef6-5f2f7325dc34 None admin] PATH_INFO: `/v3/users/3e4928183b684368857a5bcb348fc496` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:37.759893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523b9c1-1f78-4748-8ef6-5f2f7325dc34 None admin] RBAC: Authorizing `identity:get_user(user_id=3e4928183b684368857a5bcb348fc496)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:37.761846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523b9c1-1f78-4748-8ef6-5f2f7325dc34 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:37.770169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 133/264] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:37 2022] GET /identity/v3/users/3e4928183b684368857a5bcb348fc496 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:37.776448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523b9c1-1f78-4748-8ef6-5f2f7325dc34 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:37.975725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86125dab-6530-4602-ad47-6195946d8b84 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:38.005357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523b9c1-1f78-4748-8ef6-5f2f7325dc34 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:38.014318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ac758-e736-4601-8428-230a372a49ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:38.015560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ac758-e736-4601-8428-230a372a49ef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:38.016121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ac758-e736-4601-8428-230a372a49ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:38.016604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ac758-e736-4601-8428-230a372a49ef None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:38.018080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ac758-e736-4601-8428-230a372a49ef None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:38.019812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ac758-e736-4601-8428-230a372a49ef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:38.022058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 132/265] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:37 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:38.028111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ac758-e736-4601-8428-230a372a49ef None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:38.062924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:38.100620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701aa322-f0cb-4347-b5ff-4677fc560299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:38.101309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701aa322-f0cb-4347-b5ff-4677fc560299 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:38.101600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701aa322-f0cb-4347-b5ff-4677fc560299 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:38.101879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701aa322-f0cb-4347-b5ff-4677fc560299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:38.119636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:38.171968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ac758-e736-4601-8428-230a372a49ef None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:38.230275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c2223-93ba-4cef-9e9d-c547e98383a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:38.230867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c2223-93ba-4cef-9e9d-c547e98383a8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:38.231161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c2223-93ba-4cef-9e9d-c547e98383a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:38.231515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c2223-93ba-4cef-9e9d-c547e98383a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:38.232370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c2223-93ba-4cef-9e9d-c547e98383a8 None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=14787ff2bade4b019afcfdb821d8394b, user.id=3e4928183b684368857a5bcb348fc496) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:38.232687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c2223-93ba-4cef-9e9d-c547e98383a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:38.234166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c2223-93ba-4cef-9e9d-c547e98383a8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:38.249902 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 133/266] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:38 2022] GET /identity/v3/role_assignments?user.id=3e4928183b684368857a5bcb348fc496&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:38.258086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701aa322-f0cb-4347-b5ff-4677fc560299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:38.259614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701aa322-f0cb-4347-b5ff-4677fc560299 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:38.390434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 134/267] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 22 11:00:38.396643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c2223-93ba-4cef-9e9d-c547e98383a8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:38.507846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:38.515182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c2223-93ba-4cef-9e9d-c547e98383a8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:38.557283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ddb94-c6df-410b-a31f-0a46214d5516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:38.558168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ddb94-c6df-410b-a31f-0a46214d5516 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:38.558455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ddb94-c6df-410b-a31f-0a46214d5516 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:38.558736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ddb94-c6df-410b-a31f-0a46214d5516 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:38.560567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ddb94-c6df-410b-a31f-0a46214d5516 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:38.562055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ddb94-c6df-410b-a31f-0a46214d5516 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:38.563889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 134/268] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:38 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:38.570225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701aa322-f0cb-4347-b5ff-4677fc560299 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:38.618988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752b08-367c-4275-86e1-f82fa67c4b08 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:38.619350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752b08-367c-4275-86e1-f82fa67c4b08 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:38.619722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752b08-367c-4275-86e1-f82fa67c4b08 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:38.621613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 135/269] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:38.627639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef774bc1-4167-472c-9359-2f4927a27e8b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:38.627952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:38.628142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef774bc1-4167-472c-9359-2f4927a27e8b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:38.628526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef774bc1-4167-472c-9359-2f4927a27e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:38.653622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ef774bc1-4167-472c-9359-2f4927a27e8b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:38.656774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ef774bc1-4167-472c-9359-2f4927a27e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:38.761161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701aa322-f0cb-4347-b5ff-4677fc560299 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:38.803241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 136/270] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:38.825865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690214d-9896-4f33-9c68-1a20fc94b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:38.826764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690214d-9896-4f33-9c68-1a20fc94b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:38.827191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690214d-9896-4f33-9c68-1a20fc94b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:38.827625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690214d-9896-4f33-9c68-1a20fc94b7d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:38.828625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690214d-9896-4f33-9c68-1a20fc94b7d0 None admin] RBAC: Adding query filter params (role.id=2e875b35d6744573ae0287b954108209, scope.project.id=14787ff2bade4b019afcfdb821d8394b, user.id=3e4928183b684368857a5bcb348fc496) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:38.829081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690214d-9896-4f33-9c68-1a20fc94b7d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:38.830673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690214d-9896-4f33-9c68-1a20fc94b7d0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:38.840138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 135/271] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:38 2022] GET /identity/v3/role_assignments?user.id=3e4928183b684368857a5bcb348fc496&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=2e875b35d6744573ae0287b954108209 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:38.857114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef774bc1-4167-472c-9359-2f4927a27e8b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:38.968140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:38.975852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef774bc1-4167-472c-9359-2f4927a27e8b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:39.022998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60170aaf-4d2f-432b-b337-1a496f0e7403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:39.024093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170aaf-4d2f-432b-b337-1a496f0e7403 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.024524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170aaf-4d2f-432b-b337-1a496f0e7403 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.024968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60170aaf-4d2f-432b-b337-1a496f0e7403 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.032200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60170aaf-4d2f-432b-b337-1a496f0e7403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:39.033946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60170aaf-4d2f-432b-b337-1a496f0e7403 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:39.036239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 137/272] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:39.042060 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690214d-9896-4f33-9c68-1a20fc94b7d0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:39.182211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:39.185538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690214d-9896-4f33-9c68-1a20fc94b7d0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:39.227120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619b46f-2c72-4ac9-9922-420e5a7e4da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:39.227866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619b46f-2c72-4ac9-9922-420e5a7e4da8 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.228164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619b46f-2c72-4ac9-9922-420e5a7e4da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.228449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619b46f-2c72-4ac9-9922-420e5a7e4da8 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.229190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619b46f-2c72-4ac9-9922-420e5a7e4da8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:39.230663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619b46f-2c72-4ac9-9922-420e5a7e4da8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:39.256653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 136/273] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:00:39 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:39.280177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaae6c-7a86-487b-989d-25e91edc71aa None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.280786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaae6c-7a86-487b-989d-25e91edc71aa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.281284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaae6c-7a86-487b-989d-25e91edc71aa None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.282962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 138/274] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:39.289001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3bd56-5554-47a5-b8e5-a868802f9486 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.289316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3bd56-5554-47a5-b8e5-a868802f9486 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.289638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3bd56-5554-47a5-b8e5-a868802f9486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.304781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155f946-bc83-4918-8355-f13aac9f84a5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.305490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155f946-bc83-4918-8355-f13aac9f84a5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.306030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155f946-bc83-4918-8355-f13aac9f84a5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.308466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 139/275] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:39.313922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4dc3bd56-5554-47a5-b8e5-a868802f9486 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:39.315152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4dc3bd56-5554-47a5-b8e5-a868802f9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:39.319337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11522455-f00d-47a2-831c-2b9b46f6bbdb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.319970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11522455-f00d-47a2-831c-2b9b46f6bbdb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.320600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11522455-f00d-47a2-831c-2b9b46f6bbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.345176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-11522455-f00d-47a2-831c-2b9b46f6bbdb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:39.346420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-11522455-f00d-47a2-831c-2b9b46f6bbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:39.430557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 137/276] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:39.441022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 140/277] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:39.455001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3bd56-5554-47a5-b8e5-a868802f9486 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:39.464701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11522455-f00d-47a2-831c-2b9b46f6bbdb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:39.555413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:39.576524 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3bd56-5554-47a5-b8e5-a868802f9486 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:39.605690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:39.622718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11522455-f00d-47a2-831c-2b9b46f6bbdb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:39.637140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030ac63-3367-4817-b69b-135ebde3c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:39.637963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030ac63-3367-4817-b69b-135ebde3c55d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.638304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030ac63-3367-4817-b69b-135ebde3c55d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.638594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030ac63-3367-4817-b69b-135ebde3c55d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.652130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030ac63-3367-4817-b69b-135ebde3c55d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1030ac63-3367-4817-b69b-135ebde3c55d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:39.660569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:39.661472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:39.662607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 138/278] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:39.667872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030ac63-3367-4817-b69b-135ebde3c55d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:39.673590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182d5d9-e738-4e11-a5d4-f4a2c2d9ba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:39.674400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d5d9-e738-4e11-a5d4-f4a2c2d9ba7b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.674682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d5d9-e738-4e11-a5d4-f4a2c2d9ba7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.674955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d5d9-e738-4e11-a5d4-f4a2c2d9ba7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.691707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182d5d9-e738-4e11-a5d4-f4a2c2d9ba7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f182d5d9-e738-4e11-a5d4-f4a2c2d9ba7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:39.699607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:39.700914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:39.701799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 141/279] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:39.706557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182d5d9-e738-4e11-a5d4-f4a2c2d9ba7b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:39.790184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030ac63-3367-4817-b69b-135ebde3c55d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:39.819773 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182d5d9-e738-4e11-a5d4-f4a2c2d9ba7b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:39.830405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c3fd0-7403-4e76-a572-0e044937afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:39.831458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c3fd0-7403-4e76-a572-0e044937afdb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.831888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c3fd0-7403-4e76-a572-0e044937afdb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.832306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c3fd0-7403-4e76-a572-0e044937afdb None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.833197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c3fd0-7403-4e76-a572-0e044937afdb None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:39.833722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c3fd0-7403-4e76-a572-0e044937afdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:39.835530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c3fd0-7403-4e76-a572-0e044937afdb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:39.845673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 139/280] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:39.851629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c3fd0-7403-4e76-a572-0e044937afdb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:39.858919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674e0cd-78c8-4b80-bb3b-42a7ffbefa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:39.859621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674e0cd-78c8-4b80-bb3b-42a7ffbefa16 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:39.859919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674e0cd-78c8-4b80-bb3b-42a7ffbefa16 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:39.860187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674e0cd-78c8-4b80-bb3b-42a7ffbefa16 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:39.860943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674e0cd-78c8-4b80-bb3b-42a7ffbefa16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:39.861250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674e0cd-78c8-4b80-bb3b-42a7ffbefa16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:39.862658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674e0cd-78c8-4b80-bb3b-42a7ffbefa16 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:39.869117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:39.875421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 142/281] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:39.881064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674e0cd-78c8-4b80-bb3b-42a7ffbefa16 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:39.899585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:39.985836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c3fd0-7403-4e76-a572-0e044937afdb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:40.009737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674e0cd-78c8-4b80-bb3b-42a7ffbefa16 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:40.023553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f40e9-d8f0-4e50-bc79-5b1dc115a7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:40.024711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f40e9-d8f0-4e50-bc79-5b1dc115a7cd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:40.025499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f40e9-d8f0-4e50-bc79-5b1dc115a7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:40.025932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f40e9-d8f0-4e50-bc79-5b1dc115a7cd None admin] PATH_INFO: `/v3/users/0f487dfa73aa4addaa5688dcd91d020f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:40.043732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f40e9-d8f0-4e50-bc79-5b1dc115a7cd None admin] RBAC: Authorizing `identity:get_user(user_id=0f487dfa73aa4addaa5688dcd91d020f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:40.046196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f40e9-d8f0-4e50-bc79-5b1dc115a7cd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:40.048936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 140/282] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:39 2022] GET /identity/v3/users/0f487dfa73aa4addaa5688dcd91d020f => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:40.055198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f40e9-d8f0-4e50-bc79-5b1dc115a7cd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:40.057408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340349e-1117-4e1d-b064-8b9f4add2aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:40.058458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340349e-1117-4e1d-b064-8b9f4add2aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:40.058948 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340349e-1117-4e1d-b064-8b9f4add2aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:40.059468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340349e-1117-4e1d-b064-8b9f4add2aa7 None admin] PATH_INFO: `/v3/users/da57395c125b45da988aa4fd5b4d252d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:40.060963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340349e-1117-4e1d-b064-8b9f4add2aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=da57395c125b45da988aa4fd5b4d252d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:40.062649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340349e-1117-4e1d-b064-8b9f4add2aa7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:40.064892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 143/283] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:39 2022] GET /identity/v3/users/da57395c125b45da988aa4fd5b4d252d => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:40.072343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340349e-1117-4e1d-b064-8b9f4add2aa7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:40.129279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:40.179392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f40e9-d8f0-4e50-bc79-5b1dc115a7cd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:40.243521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:40.257968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4387daf-d9da-4f5b-97d8-dcb24d0d0f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:40.258647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4387daf-d9da-4f5b-97d8-dcb24d0d0f37 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:40.258936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4387daf-d9da-4f5b-97d8-dcb24d0d0f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:40.259218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4387daf-d9da-4f5b-97d8-dcb24d0d0f37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:40.260352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4387daf-d9da-4f5b-97d8-dcb24d0d0f37 None admin] RBAC: Adding query filter params (role.id=8eb9d526616b44efa3dcd0128b945ef8, user.id=0f487dfa73aa4addaa5688dcd91d020f, scope.system=all) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:40.260673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4387daf-d9da-4f5b-97d8-dcb24d0d0f37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:40.262150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4387daf-d9da-4f5b-97d8-dcb24d0d0f37 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:40.280798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 141/284] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Jul 22 11:00:40 2022] GET /identity/v3/role_assignments?user.id=0f487dfa73aa4addaa5688dcd91d020f&scope.system=all&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:40.284167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340349e-1117-4e1d-b064-8b9f4add2aa7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:40.340078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78264e2-6479-4adf-87c5-2b2014c81a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:40.340926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78264e2-6479-4adf-87c5-2b2014c81a4f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:40.341219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78264e2-6479-4adf-87c5-2b2014c81a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:40.341501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78264e2-6479-4adf-87c5-2b2014c81a4f None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:40.342734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78264e2-6479-4adf-87c5-2b2014c81a4f None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:40.344237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78264e2-6479-4adf-87c5-2b2014c81a4f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:40.345999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 144/285] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:40 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:40.352176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482e97b-9784-4fb1-8ead-7f49aac75c36 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:40.352477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482e97b-9784-4fb1-8ead-7f49aac75c36 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:40.352767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482e97b-9784-4fb1-8ead-7f49aac75c36 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:40.354209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 142/286] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:40.355199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78264e2-6479-4adf-87c5-2b2014c81a4f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:40.369197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd2104-ddf8-46b7-8244-83e23796031e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:40.369528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd2104-ddf8-46b7-8244-83e23796031e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:40.369766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd2104-ddf8-46b7-8244-83e23796031e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:40.396401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9fdd2104-ddf8-46b7-8244-83e23796031e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:40.397645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9fdd2104-ddf8-46b7-8244-83e23796031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:40.527538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78264e2-6479-4adf-87c5-2b2014c81a4f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:40.540743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 143/287] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:40.576625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd2104-ddf8-46b7-8244-83e23796031e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:40.586225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c9688-e51f-4db6-8a04-f4a507b78ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:40.587138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c9688-e51f-4db6-8a04-f4a507b78ee0 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:40.587474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c9688-e51f-4db6-8a04-f4a507b78ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:40.587757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c9688-e51f-4db6-8a04-f4a507b78ee0 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b/users/da57395c125b45da988aa4fd5b4d252d/roles/a5a3836660244b6eb47bc34d9d97907a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:40.589814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c9688-e51f-4db6-8a04-f4a507b78ee0 None admin] RBAC: Authorizing `identity:create_grant(project_id=14787ff2bade4b019afcfdb821d8394b, user_id=da57395c125b45da988aa4fd5b4d252d, role_id=a5a3836660244b6eb47bc34d9d97907a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:40.591568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c9688-e51f-4db6-8a04-f4a507b78ee0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:40.617170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 145/288] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:00:40 2022] PUT /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b/users/da57395c125b45da988aa4fd5b4d252d/roles/a5a3836660244b6eb47bc34d9d97907a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:40.682756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:40.736795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd2104-ddf8-46b7-8244-83e23796031e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:40.785749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbcca7-2b72-42cd-8e18-561d4a86bdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:40.786575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcca7-2b72-42cd-8e18-561d4a86bdfd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:40.786862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcca7-2b72-42cd-8e18-561d4a86bdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:40.787143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcca7-2b72-42cd-8e18-561d4a86bdfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:40.809342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbcca7-2b72-42cd-8e18-561d4a86bdfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4bcbcca7-2b72-42cd-8e18-561d4a86bdfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:40.817995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:40.818955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:40.819788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 144/289] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:40.826583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c9688-e51f-4db6-8a04-f4a507b78ee0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:40.861538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:40.973447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c9688-e51f-4db6-8a04-f4a507b78ee0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:41.015671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d620e8a-240b-4e84-bbdc-c02dcedcbbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:41.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d620e8a-240b-4e84-bbdc-c02dcedcbbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:41.016640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d620e8a-240b-4e84-bbdc-c02dcedcbbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:41.016922 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d620e8a-240b-4e84-bbdc-c02dcedcbbb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:41.017687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d620e8a-240b-4e84-bbdc-c02dcedcbbb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:41.018001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d620e8a-240b-4e84-bbdc-c02dcedcbbb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:41.019412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d620e8a-240b-4e84-bbdc-c02dcedcbbb1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:41.028795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 146/290] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:41.036325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbcca7-2b72-42cd-8e18-561d4a86bdfd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:41.157230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbcca7-2b72-42cd-8e18-561d4a86bdfd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:41.190587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:41.217200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa606b0-0eb4-4935-86bc-bf34846211a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:41.218122 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa606b0-0eb4-4935-86bc-bf34846211a1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:41.218415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa606b0-0eb4-4935-86bc-bf34846211a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:41.218702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa606b0-0eb4-4935-86bc-bf34846211a1 None admin] PATH_INFO: `/v3/users/f0a5288aaa1e468eb46a793a8188bebf` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:41.224037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa606b0-0eb4-4935-86bc-bf34846211a1 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a5288aaa1e468eb46a793a8188bebf)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:41.225541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa606b0-0eb4-4935-86bc-bf34846211a1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:41.227304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 145/291] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:41 2022] GET /identity/v3/users/f0a5288aaa1e468eb46a793a8188bebf => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:41.233616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d620e8a-240b-4e84-bbdc-c02dcedcbbb1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:41.318760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:41.351378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d620e8a-240b-4e84-bbdc-c02dcedcbbb1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:41.396441 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b16bd5-c88a-4dad-b9c7-6d042c55ee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:41.397098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b16bd5-c88a-4dad-b9c7-6d042c55ee4c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:41.397384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b16bd5-c88a-4dad-b9c7-6d042c55ee4c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:41.397663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b16bd5-c88a-4dad-b9c7-6d042c55ee4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:41.492435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b16bd5-c88a-4dad-b9c7-6d042c55ee4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:41.493840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b16bd5-c88a-4dad-b9c7-6d042c55ee4c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:41.576476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:41.619147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 147/292] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:41.625582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa606b0-0eb4-4935-86bc-bf34846211a1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:41.786192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa606b0-0eb4-4935-86bc-bf34846211a1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:41.833711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ce056-76e1-49d7-89bd-24956521388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:41.834593 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ce056-76e1-49d7-89bd-24956521388c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:41.834883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ce056-76e1-49d7-89bd-24956521388c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:41.835161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ce056-76e1-49d7-89bd-24956521388c None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:41.840037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ce056-76e1-49d7-89bd-24956521388c None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:41.842048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ce056-76e1-49d7-89bd-24956521388c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:41.843837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 146/293] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:41 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:41.849921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b16bd5-c88a-4dad-b9c7-6d042c55ee4c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:41.968507 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b16bd5-c88a-4dad-b9c7-6d042c55ee4c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:42.024110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006188c-6960-441d-ba98-41e098faae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:42.024780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006188c-6960-441d-ba98-41e098faae68 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.025068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006188c-6960-441d-ba98-41e098faae68 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.025346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006188c-6960-441d-ba98-41e098faae68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.026203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006188c-6960-441d-ba98-41e098faae68 None admin] RBAC: Adding query filter params (role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=14787ff2bade4b019afcfdb821d8394b, user.id=f0a5288aaa1e468eb46a793a8188bebf) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:42.026518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006188c-6960-441d-ba98-41e098faae68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:42.028003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006188c-6960-441d-ba98-41e098faae68 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:42.040785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 148/294] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:41 2022] GET /identity/v3/role_assignments?user.id=f0a5288aaa1e468eb46a793a8188bebf&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:42.285487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff77db8-7bbc-44e8-b689-29d3ab5bbae8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.285996 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff77db8-7bbc-44e8-b689-29d3ab5bbae8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.286066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff77db8-7bbc-44e8-b689-29d3ab5bbae8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.287495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 147/295] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:42.293531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318d48b-abcf-4f0f-91bf-f8df575a069b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.293879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318d48b-abcf-4f0f-91bf-f8df575a069b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.294062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318d48b-abcf-4f0f-91bf-f8df575a069b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.311617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a318d48b-abcf-4f0f-91bf-f8df575a069b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:42.312857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a318d48b-abcf-4f0f-91bf-f8df575a069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:42.363394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:42.415399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 149/296] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:42.432575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff77db8-7bbc-44e8-b689-29d3ab5bbae8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:42.492509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5608a0-8dc2-439f-ad61-a67f306829ac None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.492851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5608a0-8dc2-439f-ad61-a67f306829ac None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.493073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5608a0-8dc2-439f-ad61-a67f306829ac None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.494505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 150/297] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:42.500954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb54022-38b0-431e-b424-15c99b764eac None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.501245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb54022-38b0-431e-b424-15c99b764eac None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.501562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb54022-38b0-431e-b424-15c99b764eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.503201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:42.523071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-adb54022-38b0-431e-b424-15c99b764eac None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:42.524561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-adb54022-38b0-431e-b424-15c99b764eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:42.552061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff77db8-7bbc-44e8-b689-29d3ab5bbae8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:42.603283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dbd6c-803a-415c-b8a5-9cdf07207fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:42.603283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dbd6c-803a-415c-b8a5-9cdf07207fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.603283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dbd6c-803a-415c-b8a5-9cdf07207fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.603283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dbd6c-803a-415c-b8a5-9cdf07207fd8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.615339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dbd6c-803a-415c-b8a5-9cdf07207fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-cb1dbd6c-803a-415c-b8a5-9cdf07207fd8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:42.625130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:42.626212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:42.627164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:42.630828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 148/298] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:42.639517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dbd6c-803a-415c-b8a5-9cdf07207fd8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:42.646365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 151/299] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:42.677601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb54022-38b0-431e-b424-15c99b764eac None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:42.761838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dbd6c-803a-415c-b8a5-9cdf07207fd8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:42.781349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:42.784847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:42.789266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb54022-38b0-431e-b424-15c99b764eac None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:42.810697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a2b55-6624-49ca-ad22-68cd5dce5826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:42.811316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a2b55-6624-49ca-ad22-68cd5dce5826 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.811608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a2b55-6624-49ca-ad22-68cd5dce5826 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.811886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a2b55-6624-49ca-ad22-68cd5dce5826 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.812638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a2b55-6624-49ca-ad22-68cd5dce5826 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:42.813452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a2b55-6624-49ca-ad22-68cd5dce5826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:42.815022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a2b55-6624-49ca-ad22-68cd5dce5826 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:42.826746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 149/300] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:42.828089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655b2af-fd70-4088-9521-1aac47b5ab0f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.828541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655b2af-fd70-4088-9521-1aac47b5ab0f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.828963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655b2af-fd70-4088-9521-1aac47b5ab0f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.829230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb83ccd-b370-470b-a3a4-b147ac5bbdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:42.830576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb83ccd-b370-470b-a3a4-b147ac5bbdef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.830959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 150/301] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:42.831104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb83ccd-b370-470b-a3a4-b147ac5bbdef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.831593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb83ccd-b370-470b-a3a4-b147ac5bbdef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.832988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb83ccd-b370-470b-a3a4-b147ac5bbdef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:42.833890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655b2af-fd70-4088-9521-1aac47b5ab0f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:42.836017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb83ccd-b370-470b-a3a4-b147ac5bbdef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:42.837877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 152/302] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:42.839245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b871552-943d-4454-9ced-67ad278a39db None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.839750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b871552-943d-4454-9ced-67ad278a39db None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.840220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b871552-943d-4454-9ced-67ad278a39db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:42.857061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9b871552-943d-4454-9ced-67ad278a39db None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:42.858247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b871552-943d-4454-9ced-67ad278a39db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:42.943290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 153/303] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:42.943290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b871552-943d-4454-9ced-67ad278a39db None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:42.959696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655b2af-fd70-4088-9521-1aac47b5ab0f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:42.997768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208e0b4-e13c-41b8-82bb-c9829bc2caba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:42.998672 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208e0b4-e13c-41b8-82bb-c9829bc2caba None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:42.998959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208e0b4-e13c-41b8-82bb-c9829bc2caba None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:42.999235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208e0b4-e13c-41b8-82bb-c9829bc2caba None admin] PATH_INFO: `/v3/users/b5a909c1587e4efaa372db0867fe4c3a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.013686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208e0b4-e13c-41b8-82bb-c9829bc2caba None admin] RBAC: Authorizing `identity:get_user(user_id=b5a909c1587e4efaa372db0867fe4c3a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:43.015146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208e0b4-e13c-41b8-82bb-c9829bc2caba None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:43.022387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 151/304] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:42 2022] GET /identity/v3/users/b5a909c1587e4efaa372db0867fe4c3a => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:43.022387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208e0b4-e13c-41b8-82bb-c9829bc2caba None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:43.065810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:43.071805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:43.080633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b871552-943d-4454-9ced-67ad278a39db None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:43.119925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c2437-1f69-461c-89a9-15c4c69c7c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:43.120609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c2437-1f69-461c-89a9-15c4c69c7c22 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:43.120895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c2437-1f69-461c-89a9-15c4c69c7c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:43.121170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c2437-1f69-461c-89a9-15c4c69c7c22 None admin] PATH_INFO: `/v3/groups` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.121900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c2437-1f69-461c-89a9-15c4c69c7c22 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:43.123384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c2437-1f69-461c-89a9-15c4c69c7c22 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:43.138109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208e0b4-e13c-41b8-82bb-c9829bc2caba None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:43.144486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 154/305] 10.222.0.20 () {60 vars in 1134 bytes} [Fri Jul 22 11:00:42 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:00:43.148246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c2437-1f69-461c-89a9-15c4c69c7c22 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:43.191770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175d055-b583-4fe4-96ab-3ff47a24910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:43.192782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175d055-b583-4fe4-96ab-3ff47a24910f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:43.193286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175d055-b583-4fe4-96ab-3ff47a24910f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:43.193836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175d055-b583-4fe4-96ab-3ff47a24910f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.215170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175d055-b583-4fe4-96ab-3ff47a24910f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-7175d055-b583-4fe4-96ab-3ff47a24910f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:43.230731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:43.231817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:43.232715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:43.233313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 152/306] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:43.245042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175d055-b583-4fe4-96ab-3ff47a24910f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:43.301829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c2437-1f69-461c-89a9-15c4c69c7c22 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:43.355278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc61f5f-ce42-466d-97aa-ef4a4037959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:43.356391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc61f5f-ce42-466d-97aa-ef4a4037959d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:43.356897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc61f5f-ce42-466d-97aa-ef4a4037959d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:43.357404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc61f5f-ce42-466d-97aa-ef4a4037959d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.358971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc61f5f-ce42-466d-97aa-ef4a4037959d None admin] RBAC: Adding query filter params (role.id=a5a3836660244b6eb47bc34d9d97907a, user.id=b5a909c1587e4efaa372db0867fe4c3a, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:43.359622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc61f5f-ce42-466d-97aa-ef4a4037959d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:43.362345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc61f5f-ce42-466d-97aa-ef4a4037959d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:43.371645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 155/307] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Jul 22 11:00:43 2022] GET /identity/v3/role_assignments?user.id=b5a909c1587e4efaa372db0867fe4c3a&scope.system=all&role.id=a5a3836660244b6eb47bc34d9d97907a => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:43.380020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:43.389098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175d055-b583-4fe4-96ab-3ff47a24910f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:43.429232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d0eab-af57-4e28-bfeb-0cea57a3cf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:43.429937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d0eab-af57-4e28-bfeb-0cea57a3cf54 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:43.430225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d0eab-af57-4e28-bfeb-0cea57a3cf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:43.430509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d0eab-af57-4e28-bfeb-0cea57a3cf54 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.431330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d0eab-af57-4e28-bfeb-0cea57a3cf54 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:43.431650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d0eab-af57-4e28-bfeb-0cea57a3cf54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:43.433130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d0eab-af57-4e28-bfeb-0cea57a3cf54 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:43.443630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 153/308] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:43.452630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc61f5f-ce42-466d-97aa-ef4a4037959d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:43.480643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:43.495377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243cca4-8058-420b-b7db-a02c0654a9d5 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:43.495677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243cca4-8058-420b-b7db-a02c0654a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:43.495968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243cca4-8058-420b-b7db-a02c0654a9d5 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.498169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 154/309] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:43.503725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebfd24-87f0-4fc9-90b0-90b0a039bd70 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:43.504018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebfd24-87f0-4fc9-90b0-90b0a039bd70 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:43.504305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebfd24-87f0-4fc9-90b0-90b0a039bd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.523693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d1ebfd24-87f0-4fc9-90b0-90b0a039bd70 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:43.524945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d1ebfd24-87f0-4fc9-90b0-90b0a039bd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:43.570720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc61f5f-ce42-466d-97aa-ef4a4037959d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:43.620401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31fd25-9ffc-4f3f-992e-70fe1f87af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:43.621505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31fd25-9ffc-4f3f-992e-70fe1f87af36 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:43.622345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31fd25-9ffc-4f3f-992e-70fe1f87af36 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:43.622769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31fd25-9ffc-4f3f-992e-70fe1f87af36 None admin] PATH_INFO: `/v3/users/0f487dfa73aa4addaa5688dcd91d020f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.624503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31fd25-9ffc-4f3f-992e-70fe1f87af36 None admin] RBAC: Authorizing `identity:get_user(user_id=0f487dfa73aa4addaa5688dcd91d020f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:43.626348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31fd25-9ffc-4f3f-992e-70fe1f87af36 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:43.629032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 156/310] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:43 2022] GET /identity/v3/users/0f487dfa73aa4addaa5688dcd91d020f => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:43.632760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 155/311] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:43.634896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31fd25-9ffc-4f3f-992e-70fe1f87af36 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:43.662693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebfd24-87f0-4fc9-90b0-90b0a039bd70 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:43.753386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:43.773327 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31fd25-9ffc-4f3f-992e-70fe1f87af36 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:43.786094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:43.801952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebfd24-87f0-4fc9-90b0-90b0a039bd70 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:43.820725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72186a3f-0922-40ce-905b-1e540020c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:43.821892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72186a3f-0922-40ce-905b-1e540020c326 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:43.822188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72186a3f-0922-40ce-905b-1e540020c326 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:43.822473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72186a3f-0922-40ce-905b-1e540020c326 None admin] PATH_INFO: `/v3/system/users/0f487dfa73aa4addaa5688dcd91d020f/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.824462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72186a3f-0922-40ce-905b-1e540020c326 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0f487dfa73aa4addaa5688dcd91d020f, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:43.826742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72186a3f-0922-40ce-905b-1e540020c326 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:43.838422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 157/312] 10.222.0.20 () {58 vars in 1334 bytes} [Fri Jul 22 11:00:43 2022] PUT /identity/v3/system/users/0f487dfa73aa4addaa5688dcd91d020f/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:00:43.841342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c127962-a551-47c6-8cf5-3023eaa9af16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:43.842221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c127962-a551-47c6-8cf5-3023eaa9af16 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:43.842510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c127962-a551-47c6-8cf5-3023eaa9af16 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:43.842794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c127962-a551-47c6-8cf5-3023eaa9af16 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:43.857970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c127962-a551-47c6-8cf5-3023eaa9af16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1c127962-a551-47c6-8cf5-3023eaa9af16 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:43.866875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:43.867875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:43.868689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 156/313] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:43.876615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72186a3f-0922-40ce-905b-1e540020c326 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:44.000792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72186a3f-0922-40ce-905b-1e540020c326 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:44.036471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548df5d-b8ce-4bc8-9c7a-8058a73d9012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:44.037148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548df5d-b8ce-4bc8-9c7a-8058a73d9012 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:44.037433 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548df5d-b8ce-4bc8-9c7a-8058a73d9012 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:44.037710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548df5d-b8ce-4bc8-9c7a-8058a73d9012 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:44.038462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548df5d-b8ce-4bc8-9c7a-8058a73d9012 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:44.038768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548df5d-b8ce-4bc8-9c7a-8058a73d9012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:44.040262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548df5d-b8ce-4bc8-9c7a-8058a73d9012 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:44.049242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 158/314] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:44.054611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c127962-a551-47c6-8cf5-3023eaa9af16 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:44.162990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c127962-a551-47c6-8cf5-3023eaa9af16 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:44.197303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e916e71-faa0-49f9-9bea-c5c021e7df5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:44.198232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e916e71-faa0-49f9-9bea-c5c021e7df5f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:44.198521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e916e71-faa0-49f9-9bea-c5c021e7df5f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:44.198802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e916e71-faa0-49f9-9bea-c5c021e7df5f None admin] PATH_INFO: `/v3/users/da57395c125b45da988aa4fd5b4d252d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:44.200633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e916e71-faa0-49f9-9bea-c5c021e7df5f None admin] RBAC: Authorizing `identity:get_user(user_id=da57395c125b45da988aa4fd5b4d252d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:44.202126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e916e71-faa0-49f9-9bea-c5c021e7df5f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:44.204047 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 157/315] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:44 2022] GET /identity/v3/users/da57395c125b45da988aa4fd5b4d252d => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:44.209389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548df5d-b8ce-4bc8-9c7a-8058a73d9012 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:44.225305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:44.322046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548df5d-b8ce-4bc8-9c7a-8058a73d9012 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:44.360398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9957a9d-a0bc-4d89-992b-81e6a8dd3253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:44.361454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9957a9d-a0bc-4d89-992b-81e6a8dd3253 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:44.361990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9957a9d-a0bc-4d89-992b-81e6a8dd3253 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:44.362470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9957a9d-a0bc-4d89-992b-81e6a8dd3253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:44.435378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9957a9d-a0bc-4d89-992b-81e6a8dd3253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:44.436817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9957a9d-a0bc-4d89-992b-81e6a8dd3253 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:44.451954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:44.540226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 159/316] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:44.546525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e916e71-faa0-49f9-9bea-c5c021e7df5f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:44.566521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:44.667529 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e916e71-faa0-49f9-9bea-c5c021e7df5f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:44.703245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3772f6b-858e-40c2-a8da-b91640ae18c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:44.704067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3772f6b-858e-40c2-a8da-b91640ae18c7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:44.704356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3772f6b-858e-40c2-a8da-b91640ae18c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:44.704640 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3772f6b-858e-40c2-a8da-b91640ae18c7 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:44.705822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3772f6b-858e-40c2-a8da-b91640ae18c7 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:44.707277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3772f6b-858e-40c2-a8da-b91640ae18c7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:44.709145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 158/317] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:44 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:44.714835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9957a9d-a0bc-4d89-992b-81e6a8dd3253 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:44.816752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9957a9d-a0bc-4d89-992b-81e6a8dd3253 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:44.830222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:44.853595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3777c9-9bf6-4d6c-b395-171e6338769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:44.854300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3777c9-9bf6-4d6c-b395-171e6338769b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:44.854595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3777c9-9bf6-4d6c-b395-171e6338769b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:44.854873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3777c9-9bf6-4d6c-b395-171e6338769b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:44.855748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3777c9-9bf6-4d6c-b395-171e6338769b None admin] RBAC: Adding query filter params (role.id=a5a3836660244b6eb47bc34d9d97907a, scope.project.id=14787ff2bade4b019afcfdb821d8394b, user.id=da57395c125b45da988aa4fd5b4d252d) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:44.856610 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3777c9-9bf6-4d6c-b395-171e6338769b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:44.858082 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3777c9-9bf6-4d6c-b395-171e6338769b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:44.864106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 160/318] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:00:44 2022] GET /identity/v3/role_assignments?user.id=da57395c125b45da988aa4fd5b4d252d&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=a5a3836660244b6eb47bc34d9d97907a => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:45.398028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44f043-d5bd-472e-ab97-42cf9cf8b830 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:45.398028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44f043-d5bd-472e-ab97-42cf9cf8b830 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:45.398028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44f043-d5bd-472e-ab97-42cf9cf8b830 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:45.398028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 159/319] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:45.404720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bcc8f-a2a4-4fd2-aac8-69a7845a54fd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:45.405190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bcc8f-a2a4-4fd2-aac8-69a7845a54fd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:45.405261 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bcc8f-a2a4-4fd2-aac8-69a7845a54fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:45.422324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-587bcc8f-a2a4-4fd2-aac8-69a7845a54fd None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:45.423556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-587bcc8f-a2a4-4fd2-aac8-69a7845a54fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:45.500628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 161/320] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:45.522304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9a70-bffb-4d7c-aba7-7f86ac63fdeb None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:45.522629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9a70-bffb-4d7c-aba7-7f86ac63fdeb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:45.522873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9a70-bffb-4d7c-aba7-7f86ac63fdeb None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:45.524450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 160/321] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:45.525482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bcc8f-a2a4-4fd2-aac8-69a7845a54fd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:45.532101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7dc7f-fc39-4bac-b096-3a8f90f1befa None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:45.532577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7dc7f-fc39-4bac-b096-3a8f90f1befa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:45.532945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7dc7f-fc39-4bac-b096-3a8f90f1befa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:45.551815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-10b7dc7f-fc39-4bac-b096-3a8f90f1befa None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:45.553027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-10b7dc7f-fc39-4bac-b096-3a8f90f1befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:45.644584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 161/322] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:45.651886 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bcc8f-a2a4-4fd2-aac8-69a7845a54fd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:45.660903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:45.666813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7dc7f-fc39-4bac-b096-3a8f90f1befa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:45.683321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:45.690293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb251d-4347-4180-a482-d34e9213d78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:45.691515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb251d-4347-4180-a482-d34e9213d78b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:45.691515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb251d-4347-4180-a482-d34e9213d78b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:45.691739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb251d-4347-4180-a482-d34e9213d78b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:45.692860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb251d-4347-4180-a482-d34e9213d78b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:45.694428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb251d-4347-4180-a482-d34e9213d78b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:45.700959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 162/323] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:00:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:45.702503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb251d-4347-4180-a482-d34e9213d78b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:45.775757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7dc7f-fc39-4bac-b096-3a8f90f1befa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:45.813536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306de96-b22e-4aa2-a1dc-ecc8988b72ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:45.814100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb251d-4347-4180-a482-d34e9213d78b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:45.814463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306de96-b22e-4aa2-a1dc-ecc8988b72ee None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:45.814758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306de96-b22e-4aa2-a1dc-ecc8988b72ee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:45.815041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306de96-b22e-4aa2-a1dc-ecc8988b72ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:45.825980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306de96-b22e-4aa2-a1dc-ecc8988b72ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8306de96-b22e-4aa2-a1dc-ecc8988b72ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:45.834296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:45.835168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:45.840029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 162/324] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:45.841460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306de96-b22e-4aa2-a1dc-ecc8988b72ee None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:45.853363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e7e86-e8fd-4401-8b22-8832fb6cdeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:45.854117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e7e86-e8fd-4401-8b22-8832fb6cdeed None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:45.854469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e7e86-e8fd-4401-8b22-8832fb6cdeed None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:45.854812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e7e86-e8fd-4401-8b22-8832fb6cdeed None admin] PATH_INFO: `/v3/groups` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:45.855542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e7e86-e8fd-4401-8b22-8832fb6cdeed None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:45.857508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e7e86-e8fd-4401-8b22-8832fb6cdeed None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:45.875125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 163/325] 10.222.0.20 () {60 vars in 1134 bytes} [Fri Jul 22 11:00:45 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:00:45.928613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:45.957147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306de96-b22e-4aa2-a1dc-ecc8988b72ee None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:45.995929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e016237-f048-4dfb-bd9c-de0dd81e9372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:45.997207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e016237-f048-4dfb-bd9c-de0dd81e9372 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:45.997496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e016237-f048-4dfb-bd9c-de0dd81e9372 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:45.997771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e016237-f048-4dfb-bd9c-de0dd81e9372 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:45.998539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e016237-f048-4dfb-bd9c-de0dd81e9372 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:45.998843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e016237-f048-4dfb-bd9c-de0dd81e9372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:46.000289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e016237-f048-4dfb-bd9c-de0dd81e9372 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:46.009908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 163/326] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:46.016693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e7e86-e8fd-4401-8b22-8832fb6cdeed None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:46.052832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fce3f-275e-4365-82f2-172f77f1c591 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:46.053133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fce3f-275e-4365-82f2-172f77f1c591 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:46.053418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fce3f-275e-4365-82f2-172f77f1c591 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:46.054798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 164/327] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:00:46.057568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:46.063760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9392d60-24f6-4423-afa4-5c6ed2b12c31 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:46.064059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9392d60-24f6-4423-afa4-5c6ed2b12c31 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:46.064354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9392d60-24f6-4423-afa4-5c6ed2b12c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:46.083978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e9392d60-24f6-4423-afa4-5c6ed2b12c31 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:46.085250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e9392d60-24f6-4423-afa4-5c6ed2b12c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:46.149238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e7e86-e8fd-4401-8b22-8832fb6cdeed None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:46.177530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 165/328] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:46.189181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e4df0-cc27-409a-848d-2490848bc980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:46.189991 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e4df0-cc27-409a-848d-2490848bc980 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:46.190275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e4df0-cc27-409a-848d-2490848bc980 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:46.190548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e4df0-cc27-409a-848d-2490848bc980 None admin] PATH_INFO: `/v3/users/b5a909c1587e4efaa372db0867fe4c3a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:46.191769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e4df0-cc27-409a-848d-2490848bc980 None admin] RBAC: Authorizing `identity:get_user(user_id=b5a909c1587e4efaa372db0867fe4c3a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:46.193216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e4df0-cc27-409a-848d-2490848bc980 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:46.195013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 164/329] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:46 2022] GET /identity/v3/users/b5a909c1587e4efaa372db0867fe4c3a => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:46.200638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9392d60-24f6-4423-afa4-5c6ed2b12c31 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:46.201525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e4df0-cc27-409a-848d-2490848bc980 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:46.289668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:46.331442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9392d60-24f6-4423-afa4-5c6ed2b12c31 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:46.346277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e4df0-cc27-409a-848d-2490848bc980 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:46.347986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:46.386185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92803f-fb48-4b5f-99c8-78797af4d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:46.387911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92803f-fb48-4b5f-99c8-78797af4d13b None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:46.388451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92803f-fb48-4b5f-99c8-78797af4d13b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:46.388963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92803f-fb48-4b5f-99c8-78797af4d13b None admin] PATH_INFO: `/v3/system/users/b5a909c1587e4efaa372db0867fe4c3a/roles/a5a3836660244b6eb47bc34d9d97907a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:46.390543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fd2a3-f055-40f1-bf83-8375ab155ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:46.391114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92803f-fb48-4b5f-99c8-78797af4d13b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b5a909c1587e4efaa372db0867fe4c3a, role_id=a5a3836660244b6eb47bc34d9d97907a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:46.391447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fd2a3-f055-40f1-bf83-8375ab155ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:46.391738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fd2a3-f055-40f1-bf83-8375ab155ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:46.392021 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fd2a3-f055-40f1-bf83-8375ab155ba0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:46.394571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92803f-fb48-4b5f-99c8-78797af4d13b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:46.409650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24fd2a3-f055-40f1-bf83-8375ab155ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:46.420577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 166/330] 10.222.0.20 () {58 vars in 1334 bytes} [Fri Jul 22 11:00:46 2022] PUT /identity/v3/system/users/b5a909c1587e4efaa372db0867fe4c3a/roles/a5a3836660244b6eb47bc34d9d97907a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f24fd2a3-f055-40f1-bf83-8375ab155ba0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:46.425285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:46.426332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:46.427723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 165/331] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:46.438275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92803f-fb48-4b5f-99c8-78797af4d13b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:46.569666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92803f-fb48-4b5f-99c8-78797af4d13b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:46.620964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747311f-c17f-4c78-9953-4eb94419c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:46.623394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747311f-c17f-4c78-9953-4eb94419c9cd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:46.623696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747311f-c17f-4c78-9953-4eb94419c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:46.623992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747311f-c17f-4c78-9953-4eb94419c9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:46.626159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747311f-c17f-4c78-9953-4eb94419c9cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:46.626477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747311f-c17f-4c78-9953-4eb94419c9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:46.627928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747311f-c17f-4c78-9953-4eb94419c9cd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:46.629585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:46.639009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 167/332] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:46.645880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fd2a3-f055-40f1-bf83-8375ab155ba0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:46.759503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fd2a3-f055-40f1-bf83-8375ab155ba0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:46.829109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a7e44-9a64-49e3-840a-1e2dc27ddefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:46.829990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a7e44-9a64-49e3-840a-1e2dc27ddefe None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:46.830287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a7e44-9a64-49e3-840a-1e2dc27ddefe None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:46.830571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a7e44-9a64-49e3-840a-1e2dc27ddefe None admin] PATH_INFO: `/v3/users/0f487dfa73aa4addaa5688dcd91d020f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:46.831970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a7e44-9a64-49e3-840a-1e2dc27ddefe None admin] RBAC: Authorizing `identity:get_user(user_id=0f487dfa73aa4addaa5688dcd91d020f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:46.833517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a7e44-9a64-49e3-840a-1e2dc27ddefe None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:46.835476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 166/333] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:46 2022] GET /identity/v3/users/0f487dfa73aa4addaa5688dcd91d020f => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:46.842726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747311f-c17f-4c78-9953-4eb94419c9cd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:46.995300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747311f-c17f-4c78-9953-4eb94419c9cd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:47.039960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a414e-fb75-48a4-bf1e-d30c04ed42f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:47.040993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a414e-fb75-48a4-bf1e-d30c04ed42f6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:47.041480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a414e-fb75-48a4-bf1e-d30c04ed42f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:47.041959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a414e-fb75-48a4-bf1e-d30c04ed42f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:47.042997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a414e-fb75-48a4-bf1e-d30c04ed42f6 None admin] RBAC: Adding query filter params (role.id=8eb9d526616b44efa3dcd0128b945ef8, user.id=0f487dfa73aa4addaa5688dcd91d020f, scope.system=all) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:47.043585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a414e-fb75-48a4-bf1e-d30c04ed42f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:47.045215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a414e-fb75-48a4-bf1e-d30c04ed42f6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:47.058389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 168/334] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Jul 22 11:00:46 2022] GET /identity/v3/role_assignments?user.id=0f487dfa73aa4addaa5688dcd91d020f&scope.system=all&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:48.360722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adcaba-e890-4db6-a5f0-018657be43fe None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:48.361289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adcaba-e890-4db6-a5f0-018657be43fe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:48.361289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adcaba-e890-4db6-a5f0-018657be43fe None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:48.362919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 167/335] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:48.369216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57442414-1271-4d1d-a7e9-35d8ced4e003 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:48.369689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57442414-1271-4d1d-a7e9-35d8ced4e003 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:48.369778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57442414-1271-4d1d-a7e9-35d8ced4e003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:48.386844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-57442414-1271-4d1d-a7e9-35d8ced4e003 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:48.388629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-57442414-1271-4d1d-a7e9-35d8ced4e003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:48.444229 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:48.471466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 169/336] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:48.498696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adcaba-e890-4db6-a5f0-018657be43fe None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:48.534179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:48.629507 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adcaba-e890-4db6-a5f0-018657be43fe None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:48.673411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832510f9-3ba8-4da3-b438-5525997cefca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:48.674241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832510f9-3ba8-4da3-b438-5525997cefca None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:48.674530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832510f9-3ba8-4da3-b438-5525997cefca None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:48.674810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832510f9-3ba8-4da3-b438-5525997cefca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:48.688091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832510f9-3ba8-4da3-b438-5525997cefca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-832510f9-3ba8-4da3-b438-5525997cefca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:48.696120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:48.697070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:48.698319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 168/337] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:48.704138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57442414-1271-4d1d-a7e9-35d8ced4e003 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:48.855399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57442414-1271-4d1d-a7e9-35d8ced4e003 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:48.907285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bbe7a-bdd7-4092-ac1e-e8a2b9b6f9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:48.908058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bbe7a-bdd7-4092-ac1e-e8a2b9b6f9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:48.908352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bbe7a-bdd7-4092-ac1e-e8a2b9b6f9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:48.908699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bbe7a-bdd7-4092-ac1e-e8a2b9b6f9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:48.909606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bbe7a-bdd7-4092-ac1e-e8a2b9b6f9d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:48.909995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bbe7a-bdd7-4092-ac1e-e8a2b9b6f9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:48.911637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bbe7a-bdd7-4092-ac1e-e8a2b9b6f9d4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:48.928444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 170/338] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:48.935359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832510f9-3ba8-4da3-b438-5525997cefca None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:49.066115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:49.077460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832510f9-3ba8-4da3-b438-5525997cefca None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:49.135217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f9236-fb69-4dec-8a22-c23d607a8be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:49.136011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f9236-fb69-4dec-8a22-c23d607a8be3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:49.136357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f9236-fb69-4dec-8a22-c23d607a8be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:49.136728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f9236-fb69-4dec-8a22-c23d607a8be3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:49.249508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f9236-fb69-4dec-8a22-c23d607a8be3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:49.251295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f9236-fb69-4dec-8a22-c23d607a8be3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:49.447088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:49.460699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 169/339] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:49.466958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bbe7a-bdd7-4092-ac1e-e8a2b9b6f9d4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:49.500271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d785c11-2cb4-4679-8de8-424fb0de1dd0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:49.500561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d785c11-2cb4-4679-8de8-424fb0de1dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:49.500761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d785c11-2cb4-4679-8de8-424fb0de1dd0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:49.502339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 170/340] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:49.509401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76af9cc-3f11-4678-9c03-313d9163a373 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:49.510472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76af9cc-3f11-4678-9c03-313d9163a373 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:49.510472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76af9cc-3f11-4678-9c03-313d9163a373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:49.544504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:49.544690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e76af9cc-3f11-4678-9c03-313d9163a373 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:49.546552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e76af9cc-3f11-4678-9c03-313d9163a373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:49.636569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bbe7a-bdd7-4092-ac1e-e8a2b9b6f9d4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:49.659333 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 171/341] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:49.688171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e4fd8-9b91-45c8-8797-74ca1502726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:49.689077 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e4fd8-9b91-45c8-8797-74ca1502726c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:49.689369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e4fd8-9b91-45c8-8797-74ca1502726c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:49.689645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e4fd8-9b91-45c8-8797-74ca1502726c None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:49.691032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e4fd8-9b91-45c8-8797-74ca1502726c None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:49.692527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e4fd8-9b91-45c8-8797-74ca1502726c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:49.694595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 171/342] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:49 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:49.697129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76af9cc-3f11-4678-9c03-313d9163a373 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:49.703740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e4fd8-9b91-45c8-8797-74ca1502726c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:49.869822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e4fd8-9b91-45c8-8797-74ca1502726c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:49.883557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:49.930435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244f8df-7ed9-48e3-a4f8-a61d84c42996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:49.932221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244f8df-7ed9-48e3-a4f8-a61d84c42996 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:49.932716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244f8df-7ed9-48e3-a4f8-a61d84c42996 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:49.933196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244f8df-7ed9-48e3-a4f8-a61d84c42996 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:49.943050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244f8df-7ed9-48e3-a4f8-a61d84c42996 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:49.945057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244f8df-7ed9-48e3-a4f8-a61d84c42996 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:49.947691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 172/343] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:49 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:49.950156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5f63d-4ca9-42a0-ad1a-a3068a2fadd6 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:49.950799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5f63d-4ca9-42a0-ad1a-a3068a2fadd6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:49.951431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5f63d-4ca9-42a0-ad1a-a3068a2fadd6 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:49.953988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 173/344] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:49.955170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5f63d-4ca9-42a0-ad1a-a3068a2fadd6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:49.955338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76af9cc-3f11-4678-9c03-313d9163a373 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:50.004189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:50.039634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b387711-dd55-42e3-9469-cce1e2ad3152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:50.041172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b387711-dd55-42e3-9469-cce1e2ad3152 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.042855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b387711-dd55-42e3-9469-cce1e2ad3152 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.043516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b387711-dd55-42e3-9469-cce1e2ad3152 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.066552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b387711-dd55-42e3-9469-cce1e2ad3152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6b387711-dd55-42e3-9469-cce1e2ad3152 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:50.076126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:50.077155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:50.078001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:50.088014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 172/345] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:50.091645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb057f81-b2fa-41b7-bc2e-47f87b64b651 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.092265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb057f81-b2fa-41b7-bc2e-47f87b64b651 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.093993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb057f81-b2fa-41b7-bc2e-47f87b64b651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.119902 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eb057f81-b2fa-41b7-bc2e-47f87b64b651 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:50.121880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eb057f81-b2fa-41b7-bc2e-47f87b64b651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:50.141039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5f63d-4ca9-42a0-ad1a-a3068a2fadd6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:50.183472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2a5d2-e7ab-4c86-bca7-84cce0959126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:50.185105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2a5d2-e7ab-4c86-bca7-84cce0959126 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.185648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2a5d2-e7ab-4c86-bca7-84cce0959126 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.186124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2a5d2-e7ab-4c86-bca7-84cce0959126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.188476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2a5d2-e7ab-4c86-bca7-84cce0959126 None admin] RBAC: Adding query filter params (group.id=94cfd093cb944ba39f4d4325405d4fde, role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:50.188476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2a5d2-e7ab-4c86-bca7-84cce0959126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:50.190961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2a5d2-e7ab-4c86-bca7-84cce0959126 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:50.201584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 174/346] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:00:49 2022] GET /identity/v3/role_assignments?group.id=94cfd093cb944ba39f4d4325405d4fde&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 260 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:50.203180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5fe07-100f-485b-8a22-95a1297729fd None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.203441 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5fe07-100f-485b-8a22-95a1297729fd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.203729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5fe07-100f-485b-8a22-95a1297729fd None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.205546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 175/347] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:50.206522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5fe07-100f-485b-8a22-95a1297729fd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:50.208436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 173/348] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:50.216215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457d34-f5fc-4c21-840b-d8245d1e0c54 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.216894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457d34-f5fc-4c21-840b-d8245d1e0c54 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.217365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457d34-f5fc-4c21-840b-d8245d1e0c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.237204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-70457d34-f5fc-4c21-840b-d8245d1e0c54 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:50.238817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-70457d34-f5fc-4c21-840b-d8245d1e0c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:50.265705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:50.324728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 174/349] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:50.325882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70457d34-f5fc-4c21-840b-d8245d1e0c54 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:50.346125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5fe07-100f-485b-8a22-95a1297729fd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:50.376539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:50.401094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3606725-1126-4b9e-a2b7-46dbdb2ea581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:50.401794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3606725-1126-4b9e-a2b7-46dbdb2ea581 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.402085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3606725-1126-4b9e-a2b7-46dbdb2ea581 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.402365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3606725-1126-4b9e-a2b7-46dbdb2ea581 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.403199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3606725-1126-4b9e-a2b7-46dbdb2ea581 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:50.403527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3606725-1126-4b9e-a2b7-46dbdb2ea581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:50.405074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3606725-1126-4b9e-a2b7-46dbdb2ea581 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:50.415250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 176/350] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:50.415881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3606725-1126-4b9e-a2b7-46dbdb2ea581 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:50.451249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70457d34-f5fc-4c21-840b-d8245d1e0c54 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:50.492400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc22025-70a4-4618-9ec7-21510c68dd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:50.493673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc22025-70a4-4618-9ec7-21510c68dd8f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.494138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc22025-70a4-4618-9ec7-21510c68dd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.494601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc22025-70a4-4618-9ec7-21510c68dd8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.508094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc22025-70a4-4618-9ec7-21510c68dd8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6bc22025-70a4-4618-9ec7-21510c68dd8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:50.517527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:50.518519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:50.519387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:50.521717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 175/351] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:50.522807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc22025-70a4-4618-9ec7-21510c68dd8f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:50.534135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:50.537221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3606725-1126-4b9e-a2b7-46dbdb2ea581 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:50.582322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9fda4-da93-4be6-91ac-0bde2c3c5294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:50.583135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9fda4-da93-4be6-91ac-0bde2c3c5294 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.583459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9fda4-da93-4be6-91ac-0bde2c3c5294 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.583742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9fda4-da93-4be6-91ac-0bde2c3c5294 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.606145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9fda4-da93-4be6-91ac-0bde2c3c5294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-6ae9fda4-da93-4be6-91ac-0bde2c3c5294 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:50.614026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:50.615198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:50.616070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 177/352] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:50.616733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9fda4-da93-4be6-91ac-0bde2c3c5294 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:50.658185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:50.662488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc22025-70a4-4618-9ec7-21510c68dd8f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:50.703038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9d8c8-0d68-433b-b394-e10a636e2fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:50.704108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9d8c8-0d68-433b-b394-e10a636e2fec None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.704631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9d8c8-0d68-433b-b394-e10a636e2fec None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.705113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9d8c8-0d68-433b-b394-e10a636e2fec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.729613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9fda4-da93-4be6-91ac-0bde2c3c5294 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:50.766419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e217b2-9cbd-4d06-9632-b3477e28de8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:50.767103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e217b2-9cbd-4d06-9632-b3477e28de8c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.767434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e217b2-9cbd-4d06-9632-b3477e28de8c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.767722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e217b2-9cbd-4d06-9632-b3477e28de8c None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.768489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e217b2-9cbd-4d06-9632-b3477e28de8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:50.768804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e217b2-9cbd-4d06-9632-b3477e28de8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:50.770196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e217b2-9cbd-4d06-9632-b3477e28de8c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:50.781138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 178/353] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:50.782645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e217b2-9cbd-4d06-9632-b3477e28de8c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:50.785851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9d8c8-0d68-433b-b394-e10a636e2fec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:50.787827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9d8c8-0d68-433b-b394-e10a636e2fec None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:50.821378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:50.900598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:50.906551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 176/354] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:50.906714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e217b2-9cbd-4d06-9632-b3477e28de8c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:50.907844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9d8c8-0d68-433b-b394-e10a636e2fec None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:50.948617 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2547d-2e98-4504-aa3f-57babbab902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:50.949210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2547d-2e98-4504-aa3f-57babbab902a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:50.949544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2547d-2e98-4504-aa3f-57babbab902a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:50.949827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2547d-2e98-4504-aa3f-57babbab902a None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:50.950595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2547d-2e98-4504-aa3f-57babbab902a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:50.950908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2547d-2e98-4504-aa3f-57babbab902a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:50.952613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2547d-2e98-4504-aa3f-57babbab902a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:50.962744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 179/355] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:50.963455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2547d-2e98-4504-aa3f-57babbab902a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:51.057658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9d8c8-0d68-433b-b394-e10a636e2fec None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:51.084988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:51.101366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac961f3-b107-415b-a27e-709ad54b52b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:51.102032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac961f3-b107-415b-a27e-709ad54b52b4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.102325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac961f3-b107-415b-a27e-709ad54b52b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.102608 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac961f3-b107-415b-a27e-709ad54b52b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.111004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2547d-2e98-4504-aa3f-57babbab902a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:51.157136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5556a4-092f-4746-ad15-cb32a6751384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:51.158692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5556a4-092f-4746-ad15-cb32a6751384 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.159195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5556a4-092f-4746-ad15-cb32a6751384 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.159700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5556a4-092f-4746-ad15-cb32a6751384 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.161409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5556a4-092f-4746-ad15-cb32a6751384 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:51.163238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5556a4-092f-4746-ad15-cb32a6751384 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:51.165718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 180/356] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:50 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:51.167610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d2393-8e99-4f62-8c54-61631e07f491 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.168140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d2393-8e99-4f62-8c54-61631e07f491 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.168631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d2393-8e99-4f62-8c54-61631e07f491 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.170307 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 181/357] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:51.173662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d2393-8e99-4f62-8c54-61631e07f491 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:51.200382 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:51.205144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac961f3-b107-415b-a27e-709ad54b52b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:51.207034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac961f3-b107-415b-a27e-709ad54b52b4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:51.347976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d2393-8e99-4f62-8c54-61631e07f491 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:51.352205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 177/358] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:51.353804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b730-b88d-4023-bdbc-5617fb3561db None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.354302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b730-b88d-4023-bdbc-5617fb3561db None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.354789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b730-b88d-4023-bdbc-5617fb3561db None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.356676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 178/359] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:51.357703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417b730-b88d-4023-bdbc-5617fb3561db None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:51.392710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70129f67-9d15-46e6-836e-af50a263fe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:51.394373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70129f67-9d15-46e6-836e-af50a263fe76 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.394373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70129f67-9d15-46e6-836e-af50a263fe76 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.394622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70129f67-9d15-46e6-836e-af50a263fe76 None admin] PATH_INFO: `/v3/users/b5a909c1587e4efaa372db0867fe4c3a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.396192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70129f67-9d15-46e6-836e-af50a263fe76 None admin] RBAC: Authorizing `identity:get_user(user_id=b5a909c1587e4efaa372db0867fe4c3a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:51.397862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70129f67-9d15-46e6-836e-af50a263fe76 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:51.400305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 182/360] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:00:51 2022] GET /identity/v3/users/b5a909c1587e4efaa372db0867fe4c3a => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:51.402480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f119-9cde-4d25-bcf0-f31cce8dab41 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.408211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f119-9cde-4d25-bcf0-f31cce8dab41 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.408584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f119-9cde-4d25-bcf0-f31cce8dab41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.428279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4163f119-9cde-4d25-bcf0-f31cce8dab41 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:51.429505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4163f119-9cde-4d25-bcf0-f31cce8dab41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:51.445528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:51.499047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:51.503335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417b730-b88d-4023-bdbc-5617fb3561db None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:51.514771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 183/361] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:51.515699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163f119-9cde-4d25-bcf0-f31cce8dab41 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:51.546016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df63db-db67-42cd-a498-9c247268e028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:51.546890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df63db-db67-42cd-a498-9c247268e028 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.547181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df63db-db67-42cd-a498-9c247268e028 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.547492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df63db-db67-42cd-a498-9c247268e028 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.548687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df63db-db67-42cd-a498-9c247268e028 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:51.550178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df63db-db67-42cd-a498-9c247268e028 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:51.551856 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 179/362] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:51 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:51.553003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6a1ca-9d8b-49ad-90ea-884035be5f2d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.553291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6a1ca-9d8b-49ad-90ea-884035be5f2d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.553572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6a1ca-9d8b-49ad-90ea-884035be5f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.570683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-def6a1ca-9d8b-49ad-90ea-884035be5f2d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:51.571935 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-def6a1ca-9d8b-49ad-90ea-884035be5f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:51.650104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163f119-9cde-4d25-bcf0-f31cce8dab41 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:51.656179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 180/363] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:51.656861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6a1ca-9d8b-49ad-90ea-884035be5f2d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:51.691474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031820d-e3d2-4450-a0fc-486920f4ba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:51.692472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031820d-e3d2-4450-a0fc-486920f4ba35 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.692761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031820d-e3d2-4450-a0fc-486920f4ba35 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.693038 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031820d-e3d2-4450-a0fc-486920f4ba35 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.694711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031820d-e3d2-4450-a0fc-486920f4ba35 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:51.695867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031820d-e3d2-4450-a0fc-486920f4ba35 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:51.697604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 184/364] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:51 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:51.698387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031820d-e3d2-4450-a0fc-486920f4ba35 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:51.723631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:51.770368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:51.776043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6a1ca-9d8b-49ad-90ea-884035be5f2d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:51.813596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f77356-d2e7-4881-be25-142e9e57b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:51.814263 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f77356-d2e7-4881-be25-142e9e57b091 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.814960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f77356-d2e7-4881-be25-142e9e57b091 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.814960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f77356-d2e7-4881-be25-142e9e57b091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.818005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f77356-d2e7-4881-be25-142e9e57b091 None admin] RBAC: Adding query filter params (role.id=a5a3836660244b6eb47bc34d9d97907a, user.id=b5a909c1587e4efaa372db0867fe4c3a, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:51.818005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f77356-d2e7-4881-be25-142e9e57b091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:51.818232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031820d-e3d2-4450-a0fc-486920f4ba35 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:51.818312 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f77356-d2e7-4881-be25-142e9e57b091 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:51.836112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 181/365] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Jul 22 11:00:51 2022] GET /identity/v3/role_assignments?user.id=b5a909c1587e4efaa372db0867fe4c3a&scope.system=all&role.id=a5a3836660244b6eb47bc34d9d97907a => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:51.837154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f77356-d2e7-4881-be25-142e9e57b091 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:51.879004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef929e-9fd4-44dd-a2ad-87d66ad536d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:51.880461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef929e-9fd4-44dd-a2ad-87d66ad536d4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.880993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef929e-9fd4-44dd-a2ad-87d66ad536d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.881533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef929e-9fd4-44dd-a2ad-87d66ad536d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:51.896294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef929e-9fd4-44dd-a2ad-87d66ad536d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-27ef929e-9fd4-44dd-a2ad-87d66ad536d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:51.905526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:51.906495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:51.908885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:51.911680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 185/366] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:51.912819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef929e-9fd4-44dd-a2ad-87d66ad536d4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:51.957416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f77356-d2e7-4881-be25-142e9e57b091 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:51.997936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b3c97-b0c6-49f6-8229-550fa2f56451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:51.998616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b3c97-b0c6-49f6-8229-550fa2f56451 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:51.998952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b3c97-b0c6-49f6-8229-550fa2f56451 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:51.999241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b3c97-b0c6-49f6-8229-550fa2f56451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:52.000125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b3c97-b0c6-49f6-8229-550fa2f56451 None admin] RBAC: Adding query filter params (group.id=94cfd093cb944ba39f4d4325405d4fde, role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=14787ff2bade4b019afcfdb821d8394b) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:52.000447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b3c97-b0c6-49f6-8229-550fa2f56451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:52.001947 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b3c97-b0c6-49f6-8229-550fa2f56451 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:52.008291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 182/367] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:00:51 2022] GET /identity/v3/role_assignments?group.id=94cfd093cb944ba39f4d4325405d4fde&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:52.009144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b3c97-b0c6-49f6-8229-550fa2f56451 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:52.021795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:52.035303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:52.057980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef929e-9fd4-44dd-a2ad-87d66ad536d4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:52.120110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096806d0-f0b9-487f-9dcb-e99b0fcb6d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:52.120956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096806d0-f0b9-487f-9dcb-e99b0fcb6d94 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:52.121290 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096806d0-f0b9-487f-9dcb-e99b0fcb6d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:52.121576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096806d0-f0b9-487f-9dcb-e99b0fcb6d94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:52.129034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b3c97-b0c6-49f6-8229-550fa2f56451 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:52.143987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096806d0-f0b9-487f-9dcb-e99b0fcb6d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-096806d0-f0b9-487f-9dcb-e99b0fcb6d94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:52.156325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:52.157585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 186/368] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:52.158374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096806d0-f0b9-487f-9dcb-e99b0fcb6d94 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:52.188191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb501c05-0334-4767-a8b0-5f7a92d5f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:52.189646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb501c05-0334-4767-a8b0-5f7a92d5f119 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:52.190336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb501c05-0334-4767-a8b0-5f7a92d5f119 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:52.190874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb501c05-0334-4767-a8b0-5f7a92d5f119 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:52.192955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb501c05-0334-4767-a8b0-5f7a92d5f119 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:52.195120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb501c05-0334-4767-a8b0-5f7a92d5f119 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:52.202165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 183/369] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:52 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:52.202165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb501c05-0334-4767-a8b0-5f7a92d5f119 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:52.287726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096806d0-f0b9-487f-9dcb-e99b0fcb6d94 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:52.307139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:52.322664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb501c05-0334-4767-a8b0-5f7a92d5f119 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:52.330835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8f14a-bdca-4736-bb7d-0124a0b89593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:52.331948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8f14a-bdca-4736-bb7d-0124a0b89593 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:52.332469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8f14a-bdca-4736-bb7d-0124a0b89593 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:52.332992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8f14a-bdca-4736-bb7d-0124a0b89593 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:52.335378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8f14a-bdca-4736-bb7d-0124a0b89593 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:52.335944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8f14a-bdca-4736-bb7d-0124a0b89593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:52.337588 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8f14a-bdca-4736-bb7d-0124a0b89593 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:52.346583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:52.352150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 187/370] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:52.353894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8f14a-bdca-4736-bb7d-0124a0b89593 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:52.362792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795eb71-8ec5-45dd-8e47-68e9f96fea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:52.363508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795eb71-8ec5-45dd-8e47-68e9f96fea7e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:52.363797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795eb71-8ec5-45dd-8e47-68e9f96fea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:52.364077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795eb71-8ec5-45dd-8e47-68e9f96fea7e None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:52.365108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795eb71-8ec5-45dd-8e47-68e9f96fea7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:52.365420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795eb71-8ec5-45dd-8e47-68e9f96fea7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:52.366828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795eb71-8ec5-45dd-8e47-68e9f96fea7e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:52.380268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 184/371] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:52.380268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795eb71-8ec5-45dd-8e47-68e9f96fea7e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:52.505046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8f14a-bdca-4736-bb7d-0124a0b89593 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:52.529285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795eb71-8ec5-45dd-8e47-68e9f96fea7e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:52.553592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98c662-70b7-453d-b6e2-92c5b7210b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:52.554585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98c662-70b7-453d-b6e2-92c5b7210b2d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:52.555460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98c662-70b7-453d-b6e2-92c5b7210b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:52.555959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98c662-70b7-453d-b6e2-92c5b7210b2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:52.557874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98c662-70b7-453d-b6e2-92c5b7210b2d None admin] RBAC: Adding query filter params (group.id=94cfd093cb944ba39f4d4325405d4fde, role.id=2e875b35d6744573ae0287b954108209, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:52.558199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98c662-70b7-453d-b6e2-92c5b7210b2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:52.560490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98c662-70b7-453d-b6e2-92c5b7210b2d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:52.571309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 188/372] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:00:52 2022] GET /identity/v3/role_assignments?group.id=94cfd093cb944ba39f4d4325405d4fde&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=2e875b35d6744573ae0287b954108209 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:52.571309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98c662-70b7-453d-b6e2-92c5b7210b2d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:52.593225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6213ec9-2064-4138-b826-912b419df1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:52.593936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6213ec9-2064-4138-b826-912b419df1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:52.594224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6213ec9-2064-4138-b826-912b419df1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:52.594505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6213ec9-2064-4138-b826-912b419df1f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:52.637352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:52.723471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:52.746991 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6213ec9-2064-4138-b826-912b419df1f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:52.748580 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6213ec9-2064-4138-b826-912b419df1f1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:52.803761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98c662-70b7-453d-b6e2-92c5b7210b2d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:52.853591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af73b44-3cd2-45a5-a80e-3227454e1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:52.854604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af73b44-3cd2-45a5-a80e-3227454e1c34 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:52.855098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af73b44-3cd2-45a5-a80e-3227454e1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:52.856666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af73b44-3cd2-45a5-a80e-3227454e1c34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:52.925591 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 185/373] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:52.931339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6213ec9-2064-4138-b826-912b419df1f1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:52.951543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af73b44-3cd2-45a5-a80e-3227454e1c34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:52.953262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af73b44-3cd2-45a5-a80e-3227454e1c34 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:52.993365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:53.030227 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:53.060502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6213ec9-2064-4138-b826-912b419df1f1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:53.080942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 189/374] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:53.087151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f21d2-6ee7-463d-a3c3-49c8b9b48f43 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.087686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f21d2-6ee7-463d-a3c3-49c8b9b48f43 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.088255 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f21d2-6ee7-463d-a3c3-49c8b9b48f43 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.090746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 190/375] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:53.092289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f21d2-6ee7-463d-a3c3-49c8b9b48f43 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:53.101062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3401e9d-52d7-43c5-aae5-0aaa9f995223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:53.101930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3401e9d-52d7-43c5-aae5-0aaa9f995223 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.102212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3401e9d-52d7-43c5-aae5-0aaa9f995223 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.102488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3401e9d-52d7-43c5-aae5-0aaa9f995223 None admin] PATH_INFO: `/v3/projects/a25f160b71984d6fbacad5d6c1f3248c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.103713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3401e9d-52d7-43c5-aae5-0aaa9f995223 None admin] RBAC: Authorizing `identity:get_project(project_id=a25f160b71984d6fbacad5d6c1f3248c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:53.107755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3401e9d-52d7-43c5-aae5-0aaa9f995223 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:53.108319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 186/376] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:52 2022] GET /identity/v3/projects/a25f160b71984d6fbacad5d6c1f3248c => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:53.109561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5223704-ca96-4ebc-a67b-3b99a84dd467 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.109848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5223704-ca96-4ebc-a67b-3b99a84dd467 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.110127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5223704-ca96-4ebc-a67b-3b99a84dd467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.133586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c5223704-ca96-4ebc-a67b-3b99a84dd467 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:53.135216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c5223704-ca96-4ebc-a67b-3b99a84dd467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:53.222562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 187/377] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:53.223488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5223704-ca96-4ebc-a67b-3b99a84dd467 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:53.244525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f21d2-6ee7-463d-a3c3-49c8b9b48f43 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:53.285012 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:53.292582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce70d13-da92-490a-9420-e61b42f0e6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:53.294582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70d13-da92-490a-9420-e61b42f0e6dc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.295101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70d13-da92-490a-9420-e61b42f0e6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.295792 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70d13-da92-490a-9420-e61b42f0e6dc None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.297480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce70d13-da92-490a-9420-e61b42f0e6dc None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:53.299437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce70d13-da92-490a-9420-e61b42f0e6dc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:53.307613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 191/378] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:53 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:53.309008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce70d13-da92-490a-9420-e61b42f0e6dc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:53.343768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5223704-ca96-4ebc-a67b-3b99a84dd467 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:53.371732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:53.390237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea34428-b016-452f-a0e7-21127bf8ec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:53.391126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea34428-b016-452f-a0e7-21127bf8ec45 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.391455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea34428-b016-452f-a0e7-21127bf8ec45 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.391769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea34428-b016-452f-a0e7-21127bf8ec45 None admin] PATH_INFO: `/v3/groups/eef6b6e808144b09aec850915c343fee` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.399934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea34428-b016-452f-a0e7-21127bf8ec45 None admin] RBAC: Authorizing `identity:get_group(group_id=eef6b6e808144b09aec850915c343fee)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:53.401431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea34428-b016-452f-a0e7-21127bf8ec45 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:53.404611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 188/379] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:53 2022] GET /identity/v3/groups/eef6b6e808144b09aec850915c343fee => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:53.406773 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea34428-b016-452f-a0e7-21127bf8ec45 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:53.484717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce70d13-da92-490a-9420-e61b42f0e6dc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:53.541746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9aab5-cabf-4c34-b0d5-21b3a57623cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:53.543123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9aab5-cabf-4c34-b0d5-21b3a57623cf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.543764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9aab5-cabf-4c34-b0d5-21b3a57623cf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.544375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9aab5-cabf-4c34-b0d5-21b3a57623cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.547749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea34428-b016-452f-a0e7-21127bf8ec45 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:53.562491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9aab5-cabf-4c34-b0d5-21b3a57623cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:53.564404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b7c9aab5-cabf-4c34-b0d5-21b3a57623cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:53.577346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:53.578346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:53.579143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:53.582224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 192/380] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:53.583536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9aab5-cabf-4c34-b0d5-21b3a57623cf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:53.589047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca7be5-a995-4f0d-afde-dba2dbb618a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:53.590484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca7be5-a995-4f0d-afde-dba2dbb618a6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.590783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca7be5-a995-4f0d-afde-dba2dbb618a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.591058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca7be5-a995-4f0d-afde-dba2dbb618a6 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.593586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca7be5-a995-4f0d-afde-dba2dbb618a6 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:53.595112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca7be5-a995-4f0d-afde-dba2dbb618a6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:53.597372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 189/381] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:53 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:53.600742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca7be5-a995-4f0d-afde-dba2dbb618a6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:53.712064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:53.725982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9aab5-cabf-4c34-b0d5-21b3a57623cf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:53.731149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca7be5-a995-4f0d-afde-dba2dbb618a6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:53.770384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9edfd-1bca-4669-9cfe-7cf6cb757940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:53.771530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9edfd-1bca-4669-9cfe-7cf6cb757940 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.772065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9edfd-1bca-4669-9cfe-7cf6cb757940 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.772523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9edfd-1bca-4669-9cfe-7cf6cb757940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.773611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9edfd-1bca-4669-9cfe-7cf6cb757940 None admin] RBAC: Adding query filter params (group.id=eef6b6e808144b09aec850915c343fee, role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=a25f160b71984d6fbacad5d6c1f3248c) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:53.773815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32bfdd-5a82-44c5-ab43-3dadac820e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:53.774991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9edfd-1bca-4669-9cfe-7cf6cb757940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:53.775106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32bfdd-5a82-44c5-ab43-3dadac820e74 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.775106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32bfdd-5a82-44c5-ab43-3dadac820e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.775106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32bfdd-5a82-44c5-ab43-3dadac820e74 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.775782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32bfdd-5a82-44c5-ab43-3dadac820e74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:53.776090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32bfdd-5a82-44c5-ab43-3dadac820e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:53.777188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9edfd-1bca-4669-9cfe-7cf6cb757940 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:53.777983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32bfdd-5a82-44c5-ab43-3dadac820e74 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:53.784857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 193/382] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:00:53 2022] GET /identity/v3/role_assignments?group.id=eef6b6e808144b09aec850915c343fee&scope.project.id=a25f160b71984d6fbacad5d6c1f3248c&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:53.786075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9edfd-1bca-4669-9cfe-7cf6cb757940 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:53.788020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 190/383] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:53.796099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32bfdd-5a82-44c5-ab43-3dadac820e74 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:53.853274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:53.924323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9edfd-1bca-4669-9cfe-7cf6cb757940 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:53.937385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32bfdd-5a82-44c5-ab43-3dadac820e74 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:53.969688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c50475-cb0f-4b40-8b2a-8dc2f4d98219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:53.970736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c50475-cb0f-4b40-8b2a-8dc2f4d98219 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.971408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c50475-cb0f-4b40-8b2a-8dc2f4d98219 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.971920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c50475-cb0f-4b40-8b2a-8dc2f4d98219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.973062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c50475-cb0f-4b40-8b2a-8dc2f4d98219 None admin] RBAC: Adding query filter params (group.id=94cfd093cb944ba39f4d4325405d4fde, role.id=2e875b35d6744573ae0287b954108209, scope.project.id=14787ff2bade4b019afcfdb821d8394b) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:53.973583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c50475-cb0f-4b40-8b2a-8dc2f4d98219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:53.975511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c50475-cb0f-4b40-8b2a-8dc2f4d98219 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:53.983065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 194/384] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:00:53 2022] GET /identity/v3/role_assignments?group.id=94cfd093cb944ba39f4d4325405d4fde&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=2e875b35d6744573ae0287b954108209 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:53.984348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5b5d6-2b7a-4736-96bb-a54a4c294bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:53.985236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5b5d6-2b7a-4736-96bb-a54a4c294bbe None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:53.985522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5b5d6-2b7a-4736-96bb-a54a4c294bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:53.985801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5b5d6-2b7a-4736-96bb-a54a4c294bbe None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:53.987110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5b5d6-2b7a-4736-96bb-a54a4c294bbe None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:53.988672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5b5d6-2b7a-4736-96bb-a54a4c294bbe None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:53.990767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 191/385] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:53 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:53.996790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c50475-cb0f-4b40-8b2a-8dc2f4d98219 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:54.036560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:54.143860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c50475-cb0f-4b40-8b2a-8dc2f4d98219 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:54.187715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d31264-01fb-4fce-a52d-d46145ea7050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:54.188614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d31264-01fb-4fce-a52d-d46145ea7050 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:54.188906 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d31264-01fb-4fce-a52d-d46145ea7050 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:54.189191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d31264-01fb-4fce-a52d-d46145ea7050 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:54.190455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d31264-01fb-4fce-a52d-d46145ea7050 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:54.192013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d31264-01fb-4fce-a52d-d46145ea7050 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:54.194005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 195/386] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:53 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:54.200521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5b5d6-2b7a-4736-96bb-a54a4c294bbe None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:54.341823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5b5d6-2b7a-4736-96bb-a54a4c294bbe None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:54.355175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:54.385221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b04161-7fde-4bbc-8ea6-1105bf5eb029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:54.386110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b04161-7fde-4bbc-8ea6-1105bf5eb029 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:54.386392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b04161-7fde-4bbc-8ea6-1105bf5eb029 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:54.387185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b04161-7fde-4bbc-8ea6-1105bf5eb029 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/groups/94cfd093cb944ba39f4d4325405d4fde/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:54.392624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b04161-7fde-4bbc-8ea6-1105bf5eb029 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49e9ae4e8bc4168a9cab3aef4029701, group_id=94cfd093cb944ba39f4d4325405d4fde, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:54.394180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b04161-7fde-4bbc-8ea6-1105bf5eb029 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:54.424373 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 192/387] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 22 11:00:54 2022] PUT /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/groups/94cfd093cb944ba39f4d4325405d4fde/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:54.689955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cecc-8790-4746-9f30-f546d3f4972f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:54.690146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cecc-8790-4746-9f30-f546d3f4972f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:54.690441 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cecc-8790-4746-9f30-f546d3f4972f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:54.692269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 196/388] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:54.698486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327abba1-05af-4bd4-ad88-482d88ff26eb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:54.698997 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327abba1-05af-4bd4-ad88-482d88ff26eb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:54.699069 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327abba1-05af-4bd4-ad88-482d88ff26eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:54.724474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-327abba1-05af-4bd4-ad88-482d88ff26eb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:54.725710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-327abba1-05af-4bd4-ad88-482d88ff26eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:54.813738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 193/389] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:54.839095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282cecc-8790-4746-9f30-f546d3f4972f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:54.961914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:54.977876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282cecc-8790-4746-9f30-f546d3f4972f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:55.022932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdabe2-9ac9-4f51-ae22-c4b49c96a8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:55.024012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdabe2-9ac9-4f51-ae22-c4b49c96a8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:55.024438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdabe2-9ac9-4f51-ae22-c4b49c96a8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:55.024725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdabe2-9ac9-4f51-ae22-c4b49c96a8f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:55.037115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdabe2-9ac9-4f51-ae22-c4b49c96a8f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-68cdabe2-9ac9-4f51-ae22-c4b49c96a8f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:55.045819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:55.046737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:55.047573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:55.055591 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327abba1-05af-4bd4-ad88-482d88ff26eb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:55.059718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 197/390] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:55.181516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:55.185960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327abba1-05af-4bd4-ad88-482d88ff26eb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:55.231429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c3828-9ad1-4305-86fa-71b07a81e7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:55.232042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3828-9ad1-4305-86fa-71b07a81e7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:55.232328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3828-9ad1-4305-86fa-71b07a81e7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:55.232604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3828-9ad1-4305-86fa-71b07a81e7b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:55.233378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3828-9ad1-4305-86fa-71b07a81e7b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:55.233689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3828-9ad1-4305-86fa-71b07a81e7b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:55.235145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3828-9ad1-4305-86fa-71b07a81e7b3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:55.247963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 194/391] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:55.253716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdabe2-9ac9-4f51-ae22-c4b49c96a8f7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:55.267384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed556297-9a5f-4341-ba6d-bf9a8a867026 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:55.267690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed556297-9a5f-4341-ba6d-bf9a8a867026 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:55.267953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed556297-9a5f-4341-ba6d-bf9a8a867026 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:55.269641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 195/392] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:55.278674 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a6bb4-9501-4609-ae3f-bcb2fbd3ce92 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:55.278966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a6bb4-9501-4609-ae3f-bcb2fbd3ce92 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:55.279243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a6bb4-9501-4609-ae3f-bcb2fbd3ce92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:55.307087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f96a6bb4-9501-4609-ae3f-bcb2fbd3ce92 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:55.308406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f96a6bb4-9501-4609-ae3f-bcb2fbd3ce92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:55.370642 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdabe2-9ac9-4f51-ae22-c4b49c96a8f7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:55.421086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c1ce2-0e27-4d1e-9012-d06944ee37f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:55.421474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c1ce2-0e27-4d1e-9012-d06944ee37f0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:55.421772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c1ce2-0e27-4d1e-9012-d06944ee37f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:55.422058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c1ce2-0e27-4d1e-9012-d06944ee37f0 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:55.424003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c1ce2-0e27-4d1e-9012-d06944ee37f0 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:55.426351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c1ce2-0e27-4d1e-9012-d06944ee37f0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:55.428901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 198/393] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:55 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:55.433065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 196/394] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:55.436443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c1ce2-0e27-4d1e-9012-d06944ee37f0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:55.476365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:55.490536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a6bb4-9501-4609-ae3f-bcb2fbd3ce92 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:55.631627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:55.634553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a6bb4-9501-4609-ae3f-bcb2fbd3ce92 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:55.682171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c1ce2-0e27-4d1e-9012-d06944ee37f0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:55.691132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3c7fe-f3f9-49c3-8d75-c0d92c550b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:55.693129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3c7fe-f3f9-49c3-8d75-c0d92c550b49 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:55.693931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3c7fe-f3f9-49c3-8d75-c0d92c550b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:55.694818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3c7fe-f3f9-49c3-8d75-c0d92c550b49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:55.719144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3c7fe-f3f9-49c3-8d75-c0d92c550b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a8f3c7fe-f3f9-49c3-8d75-c0d92c550b49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:55.734195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:55.735737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:55.737091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:55.741489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 197/395] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:55.744943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49d1ae-3dc2-4e23-af13-88393ba652a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:55.746807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49d1ae-3dc2-4e23-af13-88393ba652a0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:55.748732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3c7fe-f3f9-49c3-8d75-c0d92c550b49 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:55.750376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49d1ae-3dc2-4e23-af13-88393ba652a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:55.753953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49d1ae-3dc2-4e23-af13-88393ba652a0 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:55.757540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49d1ae-3dc2-4e23-af13-88393ba652a0 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:55.765669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49d1ae-3dc2-4e23-af13-88393ba652a0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:55.768224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 199/396] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:55 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:55.773693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49d1ae-3dc2-4e23-af13-88393ba652a0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:55.937050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:55.962375 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3c7fe-f3f9-49c3-8d75-c0d92c550b49 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:55.966914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49d1ae-3dc2-4e23-af13-88393ba652a0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:56.026180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2be5d-0bb2-4d75-b4d2-25e3f5d024d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:56.028619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2be5d-0bb2-4d75-b4d2-25e3f5d024d2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:56.029624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2be5d-0bb2-4d75-b4d2-25e3f5d024d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:56.030541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2be5d-0bb2-4d75-b4d2-25e3f5d024d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:56.031881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2be5d-0bb2-4d75-b4d2-25e3f5d024d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:56.033228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99027c72-21b7-4450-9468-1ef55c4f2fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:56.033777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2be5d-0bb2-4d75-b4d2-25e3f5d024d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:56.035427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2be5d-0bb2-4d75-b4d2-25e3f5d024d2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:56.036522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99027c72-21b7-4450-9468-1ef55c4f2fdd None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:56.037007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99027c72-21b7-4450-9468-1ef55c4f2fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:56.037489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99027c72-21b7-4450-9468-1ef55c4f2fdd None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b/groups/94cfd093cb944ba39f4d4325405d4fde/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:56.041193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99027c72-21b7-4450-9468-1ef55c4f2fdd None admin] RBAC: Authorizing `identity:create_grant(project_id=14787ff2bade4b019afcfdb821d8394b, group_id=94cfd093cb944ba39f4d4325405d4fde, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:56.043727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99027c72-21b7-4450-9468-1ef55c4f2fdd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:56.054827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 198/397] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:56.061028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2be5d-0bb2-4d75-b4d2-25e3f5d024d2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:56.070678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 200/398] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 22 11:00:55 2022] PUT /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b/groups/94cfd093cb944ba39f4d4325405d4fde/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:00:56.108930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:56.235624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2be5d-0bb2-4d75-b4d2-25e3f5d024d2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:56.287648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8669c-0081-42b1-a996-f447898d62a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:56.288466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8669c-0081-42b1-a996-f447898d62a1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:56.289141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8669c-0081-42b1-a996-f447898d62a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:56.289141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8669c-0081-42b1-a996-f447898d62a1 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:56.290754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8669c-0081-42b1-a996-f447898d62a1 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:56.292362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8669c-0081-42b1-a996-f447898d62a1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:56.294775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 199/399] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:56 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:56.300737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99027c72-21b7-4450-9468-1ef55c4f2fdd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:56.432171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99027c72-21b7-4450-9468-1ef55c4f2fdd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:56.487841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22efda-add3-450c-9189-b1abbb476369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:56.488728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22efda-add3-450c-9189-b1abbb476369 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:56.489015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22efda-add3-450c-9189-b1abbb476369 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:56.489548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22efda-add3-450c-9189-b1abbb476369 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:56.490814 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22efda-add3-450c-9189-b1abbb476369 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:56.492272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22efda-add3-450c-9189-b1abbb476369 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:56.494346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 201/400] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:56 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:56.500438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8669c-0081-42b1-a996-f447898d62a1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:56.627641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:56.653109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8669c-0081-42b1-a996-f447898d62a1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:56.700184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4fae0-4fb1-462d-a941-7ff9cd3c2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:56.701435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4fae0-4fb1-462d-a941-7ff9cd3c2a62 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:56.701908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4fae0-4fb1-462d-a941-7ff9cd3c2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:56.702402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4fae0-4fb1-462d-a941-7ff9cd3c2a62 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/groups/94cfd093cb944ba39f4d4325405d4fde/roles/2e875b35d6744573ae0287b954108209` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:56.709200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4fae0-4fb1-462d-a941-7ff9cd3c2a62 None admin] RBAC: Authorizing `identity:create_grant(project_id=c49e9ae4e8bc4168a9cab3aef4029701, group_id=94cfd093cb944ba39f4d4325405d4fde, role_id=2e875b35d6744573ae0287b954108209)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:56.710910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4fae0-4fb1-462d-a941-7ff9cd3c2a62 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:56.739656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 200/401] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 22 11:00:56 2022] PUT /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701/groups/94cfd093cb944ba39f4d4325405d4fde/roles/2e875b35d6744573ae0287b954108209 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:57.030156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2afde-00f5-4889-9985-6d95b6ff6d80 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.031758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2afde-00f5-4889-9985-6d95b6ff6d80 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.032519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2afde-00f5-4889-9985-6d95b6ff6d80 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.035596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 202/402] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:00:57.040763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022bd87-734b-4ae3-9146-5876a9f33e7e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.041058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022bd87-734b-4ae3-9146-5876a9f33e7e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.041519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022bd87-734b-4ae3-9146-5876a9f33e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.064126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d022bd87-734b-4ae3-9146-5876a9f33e7e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:57.065322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d022bd87-734b-4ae3-9146-5876a9f33e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:57.115647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424fc65-e6c7-4a53-aec1-81d7eaf2b83a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.116273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424fc65-e6c7-4a53-aec1-81d7eaf2b83a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.116798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424fc65-e6c7-4a53-aec1-81d7eaf2b83a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.118402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 203/403] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:57.124650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7b87-2081-4743-8474-05351cda2dbb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.125206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7b87-2081-4743-8474-05351cda2dbb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.125713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7b87-2081-4743-8474-05351cda2dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.147650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9e7a7b87-2081-4743-8474-05351cda2dbb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:57.148664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 201/404] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:57.149528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9e7a7b87-2081-4743-8474-05351cda2dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:57.158850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:57.182601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022bd87-734b-4ae3-9146-5876a9f33e7e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:57.257708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 204/405] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:00:57.305627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:57.321948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a7b87-2081-4743-8474-05351cda2dbb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:57.367674 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022bd87-734b-4ae3-9146-5876a9f33e7e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:57.434909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e155b-03c5-4952-9130-831fdc032708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:57.436749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e155b-03c5-4952-9130-831fdc032708 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.437522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e155b-03c5-4952-9130-831fdc032708 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.438035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e155b-03c5-4952-9130-831fdc032708 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.455230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e155b-03c5-4952-9130-831fdc032708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ae9e155b-03c5-4952-9130-831fdc032708 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:57.477024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:57.478131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:57.479402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:57.480012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a7b87-2081-4743-8474-05351cda2dbb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:57.488077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 202/406] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:57.491019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dd385-f6b2-46e3-8e25-ce8114b0d802 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.491139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dd385-f6b2-46e3-8e25-ce8114b0d802 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.491472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dd385-f6b2-46e3-8e25-ce8114b0d802 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.493024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 203/407] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:57.495209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dd385-f6b2-46e3-8e25-ce8114b0d802 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:57.532934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333c9b3-0804-41cd-9d57-7ff39cc9bed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:57.534805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333c9b3-0804-41cd-9d57-7ff39cc9bed3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.535324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333c9b3-0804-41cd-9d57-7ff39cc9bed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.536170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333c9b3-0804-41cd-9d57-7ff39cc9bed3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.553085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333c9b3-0804-41cd-9d57-7ff39cc9bed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3333c9b3-0804-41cd-9d57-7ff39cc9bed3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:57.563560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:57.565085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:57.566544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 205/408] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:00:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:57.570781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52374476-954a-4a82-97ca-c1b38427bee6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.571080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52374476-954a-4a82-97ca-c1b38427bee6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.571396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52374476-954a-4a82-97ca-c1b38427bee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.578520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:57.600573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-52374476-954a-4a82-97ca-c1b38427bee6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:57.603395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-52374476-954a-4a82-97ca-c1b38427bee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:57.718644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 206/409] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:57.719806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52374476-954a-4a82-97ca-c1b38427bee6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:57.720891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dd385-f6b2-46e3-8e25-ce8114b0d802 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:57.764544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:57.774266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea0842-27c4-4ab4-915b-b705cd9057be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:57.775223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0842-27c4-4ab4-915b-b705cd9057be None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.775684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0842-27c4-4ab4-915b-b705cd9057be None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.776115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0842-27c4-4ab4-915b-b705cd9057be None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.777036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea0842-27c4-4ab4-915b-b705cd9057be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:57.777534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea0842-27c4-4ab4-915b-b705cd9057be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:57.779891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea0842-27c4-4ab4-915b-b705cd9057be None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:57.791844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 204/410] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:57.793469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea0842-27c4-4ab4-915b-b705cd9057be None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:57.858076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52374476-954a-4a82-97ca-c1b38427bee6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:57.891788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:57.899243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a7651-2ff5-4a25-b1ba-81947c89f9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:57.900253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a7651-2ff5-4a25-b1ba-81947c89f9bd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.900462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a7651-2ff5-4a25-b1ba-81947c89f9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.900893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a7651-2ff5-4a25-b1ba-81947c89f9bd None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.901754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a7651-2ff5-4a25-b1ba-81947c89f9bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:57.902360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a7651-2ff5-4a25-b1ba-81947c89f9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:57.903750 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a7651-2ff5-4a25-b1ba-81947c89f9bd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:57.918611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 207/411] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:57.918611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a7651-2ff5-4a25-b1ba-81947c89f9bd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:57.935869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea0842-27c4-4ab4-915b-b705cd9057be None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:57.972733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaa21f-dbf7-4984-b437-fa6dfc8a6101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:57.973730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaa21f-dbf7-4984-b437-fa6dfc8a6101 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:57.974105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaa21f-dbf7-4984-b437-fa6dfc8a6101 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:57.974489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaa21f-dbf7-4984-b437-fa6dfc8a6101 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:57.986842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaa21f-dbf7-4984-b437-fa6dfc8a6101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-51aaa21f-dbf7-4984-b437-fa6dfc8a6101 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:57.995174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:57.996836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:57.997825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:57.999670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 205/412] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:58.000624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaa21f-dbf7-4984-b437-fa6dfc8a6101 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:58.070310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a7651-2ff5-4a25-b1ba-81947c89f9bd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:58.072559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:58.116750 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527a759-8f07-4718-abbe-96ad878a5b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:58.118009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a759-8f07-4718-abbe-96ad878a5b10 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.118734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a759-8f07-4718-abbe-96ad878a5b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.119453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a759-8f07-4718-abbe-96ad878a5b10 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.120789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527a759-8f07-4718-abbe-96ad878a5b10 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:58.121894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaa21f-dbf7-4984-b437-fa6dfc8a6101 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:58.123244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527a759-8f07-4718-abbe-96ad878a5b10 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:58.126311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 208/413] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:57 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:58.127234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527a759-8f07-4718-abbe-96ad878a5b10 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:58.171570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1da04e-86c1-4429-bd4d-a4c17f5b80ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:58.172718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da04e-86c1-4429-bd4d-a4c17f5b80ce None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.173141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da04e-86c1-4429-bd4d-a4c17f5b80ce None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.173552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da04e-86c1-4429-bd4d-a4c17f5b80ce None admin] PATH_INFO: `/v3/groups/eef6b6e808144b09aec850915c343fee` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.175047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1da04e-86c1-4429-bd4d-a4c17f5b80ce None admin] RBAC: Authorizing `identity:get_group(group_id=eef6b6e808144b09aec850915c343fee)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:58.176713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1da04e-86c1-4429-bd4d-a4c17f5b80ce None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:58.183112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 206/414] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:57 2022] GET /identity/v3/groups/eef6b6e808144b09aec850915c343fee => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:58.184949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1da04e-86c1-4429-bd4d-a4c17f5b80ce None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:58.203549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:58.246136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527a759-8f07-4718-abbe-96ad878a5b10 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:58.280772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075db35-614f-490c-a2f8-32870b0904a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:58.281443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075db35-614f-490c-a2f8-32870b0904a4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.281730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075db35-614f-490c-a2f8-32870b0904a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.282017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075db35-614f-490c-a2f8-32870b0904a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.282760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075db35-614f-490c-a2f8-32870b0904a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:58.283071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075db35-614f-490c-a2f8-32870b0904a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:58.284475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075db35-614f-490c-a2f8-32870b0904a4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:58.293810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 209/415] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:58.294602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075db35-614f-490c-a2f8-32870b0904a4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:58.303437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1da04e-86c1-4429-bd4d-a4c17f5b80ce None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:58.328487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:58.343483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fe690-7fd7-4b7b-b878-21899a23e234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:58.344653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fe690-7fd7-4b7b-b878-21899a23e234 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.345259 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fe690-7fd7-4b7b-b878-21899a23e234 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.345769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fe690-7fd7-4b7b-b878-21899a23e234 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.347209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fe690-7fd7-4b7b-b878-21899a23e234 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:58.348983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fe690-7fd7-4b7b-b878-21899a23e234 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:58.350903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 207/416] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:58 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:58.351989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fe690-7fd7-4b7b-b878-21899a23e234 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:58.397323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075db35-614f-490c-a2f8-32870b0904a4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:58.421670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:58.432123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f423e-d125-4cca-9514-aba62d8985a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:58.432959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f423e-d125-4cca-9514-aba62d8985a1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.433246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f423e-d125-4cca-9514-aba62d8985a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.433551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f423e-d125-4cca-9514-aba62d8985a1 None admin] PATH_INFO: `/v3/projects/a25f160b71984d6fbacad5d6c1f3248c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.434901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f423e-d125-4cca-9514-aba62d8985a1 None admin] RBAC: Authorizing `identity:get_project(project_id=a25f160b71984d6fbacad5d6c1f3248c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:58.436476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f423e-d125-4cca-9514-aba62d8985a1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:58.443432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 210/417] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:58 2022] GET /identity/v3/projects/a25f160b71984d6fbacad5d6c1f3248c => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:58.443432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f423e-d125-4cca-9514-aba62d8985a1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:58.472156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fe690-7fd7-4b7b-b878-21899a23e234 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:58.508565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171c30-b504-4e2f-82b1-68cdf0007ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:58.509557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171c30-b504-4e2f-82b1-68cdf0007ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.510049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171c30-b504-4e2f-82b1-68cdf0007ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.510553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171c30-b504-4e2f-82b1-68cdf0007ec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.541699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f423e-d125-4cca-9514-aba62d8985a1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:58.573175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:58.578253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf202cd-50a4-4888-9913-ca862bc85dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:58.579145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf202cd-50a4-4888-9913-ca862bc85dde None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.579564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf202cd-50a4-4888-9913-ca862bc85dde None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.579810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf202cd-50a4-4888-9913-ca862bc85dde None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b/groups/94cfd093cb944ba39f4d4325405d4fde/roles/2e875b35d6744573ae0287b954108209` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.581912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf202cd-50a4-4888-9913-ca862bc85dde None admin] RBAC: Authorizing `identity:create_grant(project_id=14787ff2bade4b019afcfdb821d8394b, group_id=94cfd093cb944ba39f4d4325405d4fde, role_id=2e875b35d6744573ae0287b954108209)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:58.583370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf202cd-50a4-4888-9913-ca862bc85dde None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:58.585511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171c30-b504-4e2f-82b1-68cdf0007ec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:58.587186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171c30-b504-4e2f-82b1-68cdf0007ec6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:58.598418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 211/418] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 22 11:00:58 2022] PUT /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b/groups/94cfd093cb944ba39f4d4325405d4fde/roles/2e875b35d6744573ae0287b954108209 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:58.603400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf202cd-50a4-4888-9913-ca862bc85dde None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:58.699930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:58.746086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 208/419] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:58.750109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8236c0-0a99-4a0a-873f-d86e883d64da None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.750970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8236c0-0a99-4a0a-873f-d86e883d64da None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.751670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8236c0-0a99-4a0a-873f-d86e883d64da None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.754876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 209/420] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:58.756579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8236c0-0a99-4a0a-873f-d86e883d64da None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:58.770058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf202cd-50a4-4888-9913-ca862bc85dde None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:58.812326 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584d9d8-c2f6-4f41-85ee-1067e2ed556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:58.813857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d9d8-c2f6-4f41-85ee-1067e2ed556b None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.814352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d9d8-c2f6-4f41-85ee-1067e2ed556b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.814833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d9d8-c2f6-4f41-85ee-1067e2ed556b None admin] PATH_INFO: `/v3/projects/a25f160b71984d6fbacad5d6c1f3248c/groups/eef6b6e808144b09aec850915c343fee/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.818148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584d9d8-c2f6-4f41-85ee-1067e2ed556b None admin] RBAC: Authorizing `identity:create_grant(project_id=a25f160b71984d6fbacad5d6c1f3248c, group_id=eef6b6e808144b09aec850915c343fee, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:58.819907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584d9d8-c2f6-4f41-85ee-1067e2ed556b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:58.851287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 212/421] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Jul 22 11:00:58 2022] PUT /identity/v3/projects/a25f160b71984d6fbacad5d6c1f3248c/groups/eef6b6e808144b09aec850915c343fee/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:00:58.851287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25f8d-d912-460a-8e81-df6d400331bf None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:58.851287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25f8d-d912-460a-8e81-df6d400331bf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:58.851287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25f8d-d912-460a-8e81-df6d400331bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:58.867646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5cd25f8d-d912-460a-8e81-df6d400331bf None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:58.868994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5cd25f8d-d912-460a-8e81-df6d400331bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:58.917205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:58.943891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8236c0-0a99-4a0a-873f-d86e883d64da None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:58.972330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 213/422] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:59.009559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c709f68-92ad-4d4c-ab97-a1f2e628ed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:59.010578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c709f68-92ad-4d4c-ab97-a1f2e628ed2c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.010922 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c709f68-92ad-4d4c-ab97-a1f2e628ed2c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:59.011200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c709f68-92ad-4d4c-ab97-a1f2e628ed2c None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:59.013316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c709f68-92ad-4d4c-ab97-a1f2e628ed2c None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:59.014933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c709f68-92ad-4d4c-ab97-a1f2e628ed2c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:59.017008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 210/423] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:58 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:00:59.020522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25f8d-d912-460a-8e81-df6d400331bf None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:59.022714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c709f68-92ad-4d4c-ab97-a1f2e628ed2c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:59.114066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:59.154111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c709f68-92ad-4d4c-ab97-a1f2e628ed2c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:59.191368 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25f8d-d912-460a-8e81-df6d400331bf None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:59.191782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a76f96-e2c1-48da-a57e-60c8d22d5c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:59.192639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a76f96-e2c1-48da-a57e-60c8d22d5c76 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.192923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a76f96-e2c1-48da-a57e-60c8d22d5c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:59.193203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a76f96-e2c1-48da-a57e-60c8d22d5c76 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:59.194440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a76f96-e2c1-48da-a57e-60c8d22d5c76 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:59.196202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a76f96-e2c1-48da-a57e-60c8d22d5c76 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:59.198023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 211/424] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:00:59 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:59.204053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a76f96-e2c1-48da-a57e-60c8d22d5c76 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:59.232829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c18a7f-3b68-4340-8281-50a49122b5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:59.233877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c18a7f-3b68-4340-8281-50a49122b5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.234317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c18a7f-3b68-4340-8281-50a49122b5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:59.234743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c18a7f-3b68-4340-8281-50a49122b5f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:59.247517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c18a7f-3b68-4340-8281-50a49122b5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:59.255630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a4c18a7f-3b68-4340-8281-50a49122b5f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:59.255970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:59.256858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 22 11:00:59.257661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:00:59.261168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 214/425] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:00:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:00:59.276264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c18a7f-3b68-4340-8281-50a49122b5f9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:59.341152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a76f96-e2c1-48da-a57e-60c8d22d5c76 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:59.378528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29ea59-73a4-4505-b18a-0de78d97079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:59.379197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29ea59-73a4-4505-b18a-0de78d97079d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.379518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29ea59-73a4-4505-b18a-0de78d97079d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:59.379798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29ea59-73a4-4505-b18a-0de78d97079d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:59.381104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29ea59-73a4-4505-b18a-0de78d97079d None admin] RBAC: Adding query filter params (group.id=94cfd093cb944ba39f4d4325405d4fde, role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:59.381431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29ea59-73a4-4505-b18a-0de78d97079d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:59.383205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29ea59-73a4-4505-b18a-0de78d97079d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:59.392265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 212/426] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:00:59 2022] GET /identity/v3/role_assignments?group.id=94cfd093cb944ba39f4d4325405d4fde&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:59.393953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755f1d9-bc1d-417f-a31f-0e0254dd3e36 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.394249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755f1d9-bc1d-417f-a31f-0e0254dd3e36 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:59.394532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755f1d9-bc1d-417f-a31f-0e0254dd3e36 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:59.395999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c18a7f-3b68-4340-8281-50a49122b5f9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:59.396250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 213/427] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:00:59.398549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:59.403496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eda2b-229f-4dde-8ef1-ddea1e325ea0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.404143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eda2b-229f-4dde-8ef1-ddea1e325ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:59.404730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eda2b-229f-4dde-8ef1-ddea1e325ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:59.427520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-256eda2b-229f-4dde-8ef1-ddea1e325ea0 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:00:59.429515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-256eda2b-229f-4dde-8ef1-ddea1e325ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:00:59.441439 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00eb18-6f17-4e90-9887-a68f634d3cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:59.442545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00eb18-6f17-4e90-9887-a68f634d3cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.443046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00eb18-6f17-4e90-9887-a68f634d3cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:59.443567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00eb18-6f17-4e90-9887-a68f634d3cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:59.444827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00eb18-6f17-4e90-9887-a68f634d3cc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:00:59.445412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00eb18-6f17-4e90-9887-a68f634d3cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:59.447115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00eb18-6f17-4e90-9887-a68f634d3cc7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:59.458644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 215/428] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:00:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:00:59.472760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00eb18-6f17-4e90-9887-a68f634d3cc7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:59.521204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 214/429] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:00:59.556808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256eda2b-229f-4dde-8ef1-ddea1e325ea0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:59.601340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:59.676822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00eb18-6f17-4e90-9887-a68f634d3cc7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:59.692719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256eda2b-229f-4dde-8ef1-ddea1e325ea0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:59.730864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cfdf1-ea2d-40bd-bccb-d811d576d870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:59.732760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cfdf1-ea2d-40bd-bccb-d811d576d870 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.733243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cfdf1-ea2d-40bd-bccb-d811d576d870 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:59.733705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cfdf1-ea2d-40bd-bccb-d811d576d870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:59.752602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb0b7f-9c47-4477-aca2-47dd31b871fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:59.754462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb0b7f-9c47-4477-aca2-47dd31b871fb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.755093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb0b7f-9c47-4477-aca2-47dd31b871fb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:00:59.755669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb0b7f-9c47-4477-aca2-47dd31b871fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:00:59.779926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:59.783693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb0b7f-9c47-4477-aca2-47dd31b871fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-79bb0b7f-9c47-4477-aca2-47dd31b871fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:00:59.797041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:00:59.797993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:00:59.798800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 215/430] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:00:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:00:59.804097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb0b7f-9c47-4477-aca2-47dd31b871fb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:59.840903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cfdf1-ea2d-40bd-bccb-d811d576d870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:00:59.842843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cfdf1-ea2d-40bd-bccb-d811d576d870 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:00:59.920369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:00:59.953395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb0b7f-9c47-4477-aca2-47dd31b871fb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:00:59.980196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 216/431] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:00:59.989469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cfdf1-ea2d-40bd-bccb-d811d576d870 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:00:59.997934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ad134-fa21-4cc0-af91-6f1af1c478e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:00:59.999053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ad134-fa21-4cc0-af91-6f1af1c478e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:00:59.999598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ad134-fa21-4cc0-af91-6f1af1c478e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:00.000168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ad134-fa21-4cc0-af91-6f1af1c478e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:00.001147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ad134-fa21-4cc0-af91-6f1af1c478e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:00.001775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ad134-fa21-4cc0-af91-6f1af1c478e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:00.003468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ad134-fa21-4cc0-af91-6f1af1c478e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:00.014189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 216/432] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:00:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:00.019523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ad134-fa21-4cc0-af91-6f1af1c478e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:00.060978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:00.107145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cfdf1-ea2d-40bd-bccb-d811d576d870 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:00.146569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1de77-8e1a-4926-adff-29df5e9e310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:00.147435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1de77-8e1a-4926-adff-29df5e9e310c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:00.147722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1de77-8e1a-4926-adff-29df5e9e310c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:00.147999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1de77-8e1a-4926-adff-29df5e9e310c None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:00.149473 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1de77-8e1a-4926-adff-29df5e9e310c None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:00.151039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1de77-8e1a-4926-adff-29df5e9e310c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:00.153104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 217/433] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:00:59 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:00.157827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ad134-fa21-4cc0-af91-6f1af1c478e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:00.161325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1de77-8e1a-4926-adff-29df5e9e310c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:00.200351 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39018ea0-cbdb-4b97-b576-d8d96829ace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:00.201675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39018ea0-cbdb-4b97-b576-d8d96829ace3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:00.202150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39018ea0-cbdb-4b97-b576-d8d96829ace3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:00.202843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39018ea0-cbdb-4b97-b576-d8d96829ace3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:00.218046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:00.309325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1de77-8e1a-4926-adff-29df5e9e310c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:00.317035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39018ea0-cbdb-4b97-b576-d8d96829ace3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:00.318241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39018ea0-cbdb-4b97-b576-d8d96829ace3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:00.365111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:00.373924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58753e-91c8-48a4-bfa9-20ad09089e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:00.375124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58753e-91c8-48a4-bfa9-20ad09089e30 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:00.375632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58753e-91c8-48a4-bfa9-20ad09089e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:00.376105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58753e-91c8-48a4-bfa9-20ad09089e30 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:00.377534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58753e-91c8-48a4-bfa9-20ad09089e30 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:00.379202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58753e-91c8-48a4-bfa9-20ad09089e30 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:00.381445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 218/434] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:01:00 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:00.387590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58753e-91c8-48a4-bfa9-20ad09089e30 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:00.449800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 217/435] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:00.454399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39018ea0-cbdb-4b97-b576-d8d96829ace3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:00.514978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:00.523765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58753e-91c8-48a4-bfa9-20ad09089e30 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:00.561948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1434a2b-ef71-40a6-8f65-9e52a6bab366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:00.562655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1434a2b-ef71-40a6-8f65-9e52a6bab366 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:00.562939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1434a2b-ef71-40a6-8f65-9e52a6bab366 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:00.563594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1434a2b-ef71-40a6-8f65-9e52a6bab366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:00.564440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1434a2b-ef71-40a6-8f65-9e52a6bab366 None admin] RBAC: Adding query filter params (group.id=94cfd093cb944ba39f4d4325405d4fde, role.id=8eb9d526616b44efa3dcd0128b945ef8, scope.project.id=14787ff2bade4b019afcfdb821d8394b) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:00.564799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1434a2b-ef71-40a6-8f65-9e52a6bab366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:00.566254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1434a2b-ef71-40a6-8f65-9e52a6bab366 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:00.573463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 219/436] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:01:00 2022] GET /identity/v3/role_assignments?group.id=94cfd093cb944ba39f4d4325405d4fde&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=8eb9d526616b44efa3dcd0128b945ef8 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:00.591280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39018ea0-cbdb-4b97-b576-d8d96829ace3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:00.633692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7ec13-bc8f-4e91-8bfa-6ae57c3456d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:00.634582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7ec13-bc8f-4e91-8bfa-6ae57c3456d8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:00.634866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7ec13-bc8f-4e91-8bfa-6ae57c3456d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:00.635141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7ec13-bc8f-4e91-8bfa-6ae57c3456d8 None admin] PATH_INFO: `/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:00.636523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7ec13-bc8f-4e91-8bfa-6ae57c3456d8 None admin] RBAC: Authorizing `identity:get_project(project_id=c49e9ae4e8bc4168a9cab3aef4029701)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:00.638030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7ec13-bc8f-4e91-8bfa-6ae57c3456d8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:00.644709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 218/437] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:01:00 2022] GET /identity/v3/projects/c49e9ae4e8bc4168a9cab3aef4029701 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:00.646206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1434a2b-ef71-40a6-8f65-9e52a6bab366 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:00.725575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:00.793611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1434a2b-ef71-40a6-8f65-9e52a6bab366 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:00.853422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5602ec7-3997-4588-a107-c24f3d3ca330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:00.854791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5602ec7-3997-4588-a107-c24f3d3ca330 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:00.855317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5602ec7-3997-4588-a107-c24f3d3ca330 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:00.855775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5602ec7-3997-4588-a107-c24f3d3ca330 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:00.857911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5602ec7-3997-4588-a107-c24f3d3ca330 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:00.860661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5602ec7-3997-4588-a107-c24f3d3ca330 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:00.866834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 220/438] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:01:00 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:00.875085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7ec13-bc8f-4e91-8bfa-6ae57c3456d8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:00.978189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbd958-ce02-4207-93d7-5649ef23a0cf None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:00.978513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbd958-ce02-4207-93d7-5649ef23a0cf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:00.978775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbd958-ce02-4207-93d7-5649ef23a0cf None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:00.980720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 221/439] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:00.985858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d999f2-1587-4e5d-b42d-50bf809b570e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:00.986192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d999f2-1587-4e5d-b42d-50bf809b570e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:00.986441 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d999f2-1587-4e5d-b42d-50bf809b570e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:00.996835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7ec13-bc8f-4e91-8bfa-6ae57c3456d8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:01.008131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a2d999f2-1587-4e5d-b42d-50bf809b570e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:01.009716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a2d999f2-1587-4e5d-b42d-50bf809b570e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:01.012163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:01.038908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab5d63-ca90-45e0-9a54-f989c958240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:01.039602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab5d63-ca90-45e0-9a54-f989c958240b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.039881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab5d63-ca90-45e0-9a54-f989c958240b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.040159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab5d63-ca90-45e0-9a54-f989c958240b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.041013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab5d63-ca90-45e0-9a54-f989c958240b None admin] RBAC: Adding query filter params (group.id=94cfd093cb944ba39f4d4325405d4fde, role.id=2e875b35d6744573ae0287b954108209, scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:01.041493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab5d63-ca90-45e0-9a54-f989c958240b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:01.043465 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab5d63-ca90-45e0-9a54-f989c958240b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:01.058037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 219/440] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:01:00 2022] GET /identity/v3/role_assignments?group.id=94cfd093cb944ba39f4d4325405d4fde&scope.project.id=c49e9ae4e8bc4168a9cab3aef4029701&role.id=2e875b35d6744573ae0287b954108209 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:01.100391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 222/441] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:01:01.125863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab5d63-ca90-45e0-9a54-f989c958240b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:01.246338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab5d63-ca90-45e0-9a54-f989c958240b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:01.267567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9c6da-b877-421e-8430-5d4f872693fe None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.268059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9c6da-b877-421e-8430-5d4f872693fe None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.268148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9c6da-b877-421e-8430-5d4f872693fe None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.269658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 223/442] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:01.274956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8ad1b-e48d-4ae0-9531-c536f5e88943 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.274956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8ad1b-e48d-4ae0-9531-c536f5e88943 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.275159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8ad1b-e48d-4ae0-9531-c536f5e88943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.284758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378610dc-ff8e-495a-aff2-18ae0d2f187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:01.285572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378610dc-ff8e-495a-aff2-18ae0d2f187e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.285859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378610dc-ff8e-495a-aff2-18ae0d2f187e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.286137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378610dc-ff8e-495a-aff2-18ae0d2f187e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.294042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-04a8ad1b-e48d-4ae0-9531-c536f5e88943 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:01.295378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-04a8ad1b-e48d-4ae0-9531-c536f5e88943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:01.298612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378610dc-ff8e-495a-aff2-18ae0d2f187e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-378610dc-ff8e-495a-aff2-18ae0d2f187e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:01.306705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:01.307624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:01.308468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 220/443] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:01.314454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378610dc-ff8e-495a-aff2-18ae0d2f187e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:01.352869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:01.366694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:01.382191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 224/444] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:01:01.406196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8ad1b-e48d-4ae0-9531-c536f5e88943 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:01.436350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378610dc-ff8e-495a-aff2-18ae0d2f187e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:01.475579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15553d5-53fa-403e-bcb5-4482bc46381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:01.476256 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15553d5-53fa-403e-bcb5-4482bc46381a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.476550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15553d5-53fa-403e-bcb5-4482bc46381a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.476826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15553d5-53fa-403e-bcb5-4482bc46381a None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.477652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15553d5-53fa-403e-bcb5-4482bc46381a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:01.477926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15553d5-53fa-403e-bcb5-4482bc46381a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:01.479361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15553d5-53fa-403e-bcb5-4482bc46381a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:01.488606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 221/445] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:01.494437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15553d5-53fa-403e-bcb5-4482bc46381a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:01.521829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8ad1b-e48d-4ae0-9531-c536f5e88943 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:01.569883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffec391-2357-4b9d-a708-097e937041ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:01.570689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffec391-2357-4b9d-a708-097e937041ef None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.570981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffec391-2357-4b9d-a708-097e937041ef None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.571276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffec391-2357-4b9d-a708-097e937041ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.585253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffec391-2357-4b9d-a708-097e937041ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-2ffec391-2357-4b9d-a708-097e937041ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:01.593780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:01.594699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:01.595531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:01.596757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 225/446] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:01.602537 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffec391-2357-4b9d-a708-097e937041ef None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:01.607878 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:01.613793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15553d5-53fa-403e-bcb5-4482bc46381a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:01.648936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c060c-aa22-4202-8ac1-09900b97feaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:01.649515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c060c-aa22-4202-8ac1-09900b97feaa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.649806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c060c-aa22-4202-8ac1-09900b97feaa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.650091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c060c-aa22-4202-8ac1-09900b97feaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.657462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:01.703071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffec391-2357-4b9d-a708-097e937041ef None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:01.716909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c060c-aa22-4202-8ac1-09900b97feaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:01.718387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c060c-aa22-4202-8ac1-09900b97feaa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:01.738285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fc272-6036-4d1b-9c1b-e7def7493ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:01.739602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fc272-6036-4d1b-9c1b-e7def7493ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.739897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fc272-6036-4d1b-9c1b-e7def7493ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.740183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fc272-6036-4d1b-9c1b-e7def7493ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.740936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fc272-6036-4d1b-9c1b-e7def7493ff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:01.741250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fc272-6036-4d1b-9c1b-e7def7493ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:01.742634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fc272-6036-4d1b-9c1b-e7def7493ff1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:01.750871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 226/447] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:01.756323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fc272-6036-4d1b-9c1b-e7def7493ff1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:01.810193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:01.817087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 222/448] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 22 11:01:01.822625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c060c-aa22-4202-8ac1-09900b97feaa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:01.854879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fc272-6036-4d1b-9c1b-e7def7493ff1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:01.869654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:01.900346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236b934-60f6-4b9a-aaa3-3017cd1578ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:01.900928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236b934-60f6-4b9a-aaa3-3017cd1578ee None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.901217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236b934-60f6-4b9a-aaa3-3017cd1578ee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.901496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236b934-60f6-4b9a-aaa3-3017cd1578ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.928913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c060c-aa22-4202-8ac1-09900b97feaa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:01.965380 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc8240-951c-4990-bb11-e327983c5802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:01.966254 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc8240-951c-4990-bb11-e327983c5802 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:01.966536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc8240-951c-4990-bb11-e327983c5802 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:01.966812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc8240-951c-4990-bb11-e327983c5802 None admin] PATH_INFO: `/v3/projects/14787ff2bade4b019afcfdb821d8394b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:01.968053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc8240-951c-4990-bb11-e327983c5802 None admin] RBAC: Authorizing `identity:get_project(project_id=14787ff2bade4b019afcfdb821d8394b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:01.969527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc8240-951c-4990-bb11-e327983c5802 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:01.971244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 223/449] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:01:01 2022] GET /identity/v3/projects/14787ff2bade4b019afcfdb821d8394b => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:01.979896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc8240-951c-4990-bb11-e327983c5802 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:01.980049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236b934-60f6-4b9a-aaa3-3017cd1578ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:01.981387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236b934-60f6-4b9a-aaa3-3017cd1578ee None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:02.060959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:02.101633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc8240-951c-4990-bb11-e327983c5802 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:02.102812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 227/450] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 22 11:01:02.108402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236b934-60f6-4b9a-aaa3-3017cd1578ee None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:02.123477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:02.138483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402d9f6-691a-4f97-b52a-28d48534db93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:02.139376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402d9f6-691a-4f97-b52a-28d48534db93 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:02.139661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402d9f6-691a-4f97-b52a-28d48534db93 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:02.139938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402d9f6-691a-4f97-b52a-28d48534db93 None admin] PATH_INFO: `/v3/groups/94cfd093cb944ba39f4d4325405d4fde` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:02.141124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402d9f6-691a-4f97-b52a-28d48534db93 None admin] RBAC: Authorizing `identity:get_group(group_id=94cfd093cb944ba39f4d4325405d4fde)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:02.142589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402d9f6-691a-4f97-b52a-28d48534db93 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:02.144568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 224/451] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:01:01 2022] GET /identity/v3/groups/94cfd093cb944ba39f4d4325405d4fde => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:02.150100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402d9f6-691a-4f97-b52a-28d48534db93 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:02.216206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236b934-60f6-4b9a-aaa3-3017cd1578ee None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:02.252446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c17e0b-6dd4-466d-bac4-a5bfcf1fbd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:02.253316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c17e0b-6dd4-466d-bac4-a5bfcf1fbd6a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:02.253607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c17e0b-6dd4-466d-bac4-a5bfcf1fbd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:02.253888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c17e0b-6dd4-466d-bac4-a5bfcf1fbd6a None admin] PATH_INFO: `/v3/projects/a25f160b71984d6fbacad5d6c1f3248c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:02.255082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c17e0b-6dd4-466d-bac4-a5bfcf1fbd6a None admin] RBAC: Authorizing `identity:get_project(project_id=a25f160b71984d6fbacad5d6c1f3248c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:02.256497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402d9f6-691a-4f97-b52a-28d48534db93 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:02.256604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c17e0b-6dd4-466d-bac4-a5bfcf1fbd6a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:02.258224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 228/452] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:01:02 2022] GET /identity/v3/projects/a25f160b71984d6fbacad5d6c1f3248c => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:02.264309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c17e0b-6dd4-466d-bac4-a5bfcf1fbd6a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:02.287285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:02.293560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d4da-77c6-4986-b2a9-67fe15a8da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:02.294223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d4da-77c6-4986-b2a9-67fe15a8da55 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:02.294508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d4da-77c6-4986-b2a9-67fe15a8da55 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:02.294796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d4da-77c6-4986-b2a9-67fe15a8da55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:02.295779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6d4da-77c6-4986-b2a9-67fe15a8da55 None admin] RBAC: Adding query filter params (group.id=94cfd093cb944ba39f4d4325405d4fde, role.id=2e875b35d6744573ae0287b954108209, scope.project.id=14787ff2bade4b019afcfdb821d8394b) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:02.296097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6d4da-77c6-4986-b2a9-67fe15a8da55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:02.297543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6d4da-77c6-4986-b2a9-67fe15a8da55 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:02.303597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 225/453] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:01:02 2022] GET /identity/v3/role_assignments?group.id=94cfd093cb944ba39f4d4325405d4fde&scope.project.id=14787ff2bade4b019afcfdb821d8394b&role.id=2e875b35d6744573ae0287b954108209 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:02.349293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:02.371465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c17e0b-6dd4-466d-bac4-a5bfcf1fbd6a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:02.407967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd77077-883c-4841-aae8-ffd585724a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:02.409151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd77077-883c-4841-aae8-ffd585724a30 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:02.409691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd77077-883c-4841-aae8-ffd585724a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:02.410217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd77077-883c-4841-aae8-ffd585724a30 None admin] PATH_INFO: `/v3/groups/eef6b6e808144b09aec850915c343fee` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:02.411623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd77077-883c-4841-aae8-ffd585724a30 None admin] RBAC: Authorizing `identity:get_group(group_id=eef6b6e808144b09aec850915c343fee)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:02.413117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd77077-883c-4841-aae8-ffd585724a30 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:02.414810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 229/454] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Jul 22 11:01:02 2022] GET /identity/v3/groups/eef6b6e808144b09aec850915c343fee => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:02.421463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d4da-77c6-4986-b2a9-67fe15a8da55 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:02.525132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d4da-77c6-4986-b2a9-67fe15a8da55 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:02.561355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07929f73-229e-475f-9b20-ebda6ca17e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:02.562011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07929f73-229e-475f-9b20-ebda6ca17e33 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:02.562300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07929f73-229e-475f-9b20-ebda6ca17e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:02.562581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07929f73-229e-475f-9b20-ebda6ca17e33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:02.563471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07929f73-229e-475f-9b20-ebda6ca17e33 None admin] RBAC: Adding query filter params (group.id=eef6b6e808144b09aec850915c343fee, role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=a25f160b71984d6fbacad5d6c1f3248c) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:02.563789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07929f73-229e-475f-9b20-ebda6ca17e33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:02.565509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07929f73-229e-475f-9b20-ebda6ca17e33 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:02.571698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 226/455] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Jul 22 11:01:02 2022] GET /identity/v3/role_assignments?group.id=eef6b6e808144b09aec850915c343fee&scope.project.id=a25f160b71984d6fbacad5d6c1f3248c&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:05.284706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e365e2f-27a5-4360-89af-5a606e3d3513 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.285223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e365e2f-27a5-4360-89af-5a606e3d3513 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.285297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e365e2f-27a5-4360-89af-5a606e3d3513 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.286884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 230/456] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:05.293029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202379e7-255d-419f-b6ea-f64b028a1891 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.293468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202379e7-255d-419f-b6ea-f64b028a1891 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.293649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202379e7-255d-419f-b6ea-f64b028a1891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.316295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-202379e7-255d-419f-b6ea-f64b028a1891 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:05.317529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-202379e7-255d-419f-b6ea-f64b028a1891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:05.369018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1ba88-d3dc-47a9-af15-92edb5a9d9be None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.369353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1ba88-d3dc-47a9-af15-92edb5a9d9be None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.369596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1ba88-d3dc-47a9-af15-92edb5a9d9be None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.371805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 231/457] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:05.377270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297f5c8-8e74-448b-8859-964cadfdaf1a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.377868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297f5c8-8e74-448b-8859-964cadfdaf1a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.378341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297f5c8-8e74-448b-8859-964cadfdaf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.395714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:05.403544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b297f5c8-8e74-448b-8859-964cadfdaf1a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:05.405051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b297f5c8-8e74-448b-8859-964cadfdaf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:05.426116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 227/458] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:01:05.431639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e754dc4-0221-4fbc-b28f-da5a2a554ea2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.432133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e754dc4-0221-4fbc-b28f-da5a2a554ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.432582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e754dc4-0221-4fbc-b28f-da5a2a554ea2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.435769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 228/459] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:05.443285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e07eb3-ed79-4b6d-a948-cce1c9d38432 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.443285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e07eb3-ed79-4b6d-a948-cce1c9d38432 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.443285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e07eb3-ed79-4b6d-a948-cce1c9d38432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.470006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-37e07eb3-ed79-4b6d-a948-cce1c9d38432 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:05.472143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-37e07eb3-ed79-4b6d-a948-cce1c9d38432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:05.502350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 232/460] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:05.503224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297f5c8-8e74-448b-8859-964cadfdaf1a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:05.563410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 229/461] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:01:05.564318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e07eb3-ed79-4b6d-a948-cce1c9d38432 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:05.645345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:05.664975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297f5c8-8e74-448b-8859-964cadfdaf1a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:05.693717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e07eb3-ed79-4b6d-a948-cce1c9d38432 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:05.702895 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed64b23-464c-4ee7-b9ff-05d824044f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:05.703815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed64b23-464c-4ee7-b9ff-05d824044f1a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.704024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed64b23-464c-4ee7-b9ff-05d824044f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.704304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed64b23-464c-4ee7-b9ff-05d824044f1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.715892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed64b23-464c-4ee7-b9ff-05d824044f1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:05.716414 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:05.717035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed64b23-464c-4ee7-b9ff-05d824044f1a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:05.731737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8082ddc-6606-4942-8b43-c5e98f4cbf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:05.732539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8082ddc-6606-4942-8b43-c5e98f4cbf84 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.732824 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8082ddc-6606-4942-8b43-c5e98f4cbf84 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.733159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8082ddc-6606-4942-8b43-c5e98f4cbf84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-bed64b23-464c-4ee7-b9ff-05d824044f1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:05.736364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:05.737231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:05.738501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:05.738501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:05.738501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:05.738501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:05.742867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 233/462] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:05.743758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed64b23-464c-4ee7-b9ff-05d824044f1a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:05.744516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8082ddc-6606-4942-8b43-c5e98f4cbf84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:05.746112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8082ddc-6606-4942-8b43-c5e98f4cbf84 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f8082ddc-6606-4942-8b43-c5e98f4cbf84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:05.768827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:05.769762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:05.771023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:05.771023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:05.771023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:05.771023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:05.771023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 230/463] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:05.771719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184e027-c1ab-4647-b522-6e2bfce9d749 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.772011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184e027-c1ab-4647-b522-6e2bfce9d749 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.772400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184e027-c1ab-4647-b522-6e2bfce9d749 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.775400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 231/464] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:05.775400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184e027-c1ab-4647-b522-6e2bfce9d749 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:05.862633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed64b23-464c-4ee7-b9ff-05d824044f1a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:05.888456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184e027-c1ab-4647-b522-6e2bfce9d749 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:05.894911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:05.901382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1ac6a-cedd-48c2-b743-1f95b17a7448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:05.902694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1ac6a-cedd-48c2-b743-1f95b17a7448 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.903366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1ac6a-cedd-48c2-b743-1f95b17a7448 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.903761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1ac6a-cedd-48c2-b743-1f95b17a7448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.915234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1ac6a-cedd-48c2-b743-1f95b17a7448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:05.916706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1ac6a-cedd-48c2-b743-1f95b17a7448 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:05.931566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca71137-4e62-44fb-8ca0-f2a2827a73c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:05.932243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca71137-4e62-44fb-8ca0-f2a2827a73c1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.932535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca71137-4e62-44fb-8ca0-f2a2827a73c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.932821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca71137-4e62-44fb-8ca0-f2a2827a73c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.933589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca71137-4e62-44fb-8ca0-f2a2827a73c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:05.933901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca71137-4e62-44fb-8ca0-f2a2827a73c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:05.935420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca71137-4e62-44fb-8ca0-f2a2827a73c1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d5e1ac6a-cedd-48c2-b743-1f95b17a7448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:05.936256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:05.937068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:05.937904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:05.937904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:05.937904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:05.937904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:05.937904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 234/465] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:05.938750 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1ac6a-cedd-48c2-b743-1f95b17a7448 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:05.956007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 232/466] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:05.957378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c587f6-d5f7-4225-b7c2-c8633b0ae051 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:05.957679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c587f6-d5f7-4225-b7c2-c8633b0ae051 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:05.957963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c587f6-d5f7-4225-b7c2-c8633b0ae051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:05.975883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-66c587f6-d5f7-4225-b7c2-c8633b0ae051 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:05.977095 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66c587f6-d5f7-4225-b7c2-c8633b0ae051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:06.012285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:06.069044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 233/467] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:06.069044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c587f6-d5f7-4225-b7c2-c8633b0ae051 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:06.075172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1ac6a-cedd-48c2-b743-1f95b17a7448 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:06.114655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b75f68-d2aa-4c91-a539-5b677fbbac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:06.115247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75f68-d2aa-4c91-a539-5b677fbbac6a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.115567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75f68-d2aa-4c91-a539-5b677fbbac6a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.115849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75f68-d2aa-4c91-a539-5b677fbbac6a None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.116579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b75f68-d2aa-4c91-a539-5b677fbbac6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:06.116894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b75f68-d2aa-4c91-a539-5b677fbbac6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:06.118283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b75f68-d2aa-4c91-a539-5b677fbbac6a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:06.130666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 235/468] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:06.131313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b75f68-d2aa-4c91-a539-5b677fbbac6a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:06.177317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c587f6-d5f7-4225-b7c2-c8633b0ae051 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:06.184499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:06.218988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94a63-5571-4afc-8d6f-4747f3266abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:06.220046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94a63-5571-4afc-8d6f-4747f3266abc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.220513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94a63-5571-4afc-8d6f-4747f3266abc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.220968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94a63-5571-4afc-8d6f-4747f3266abc None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.222168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a94a63-5571-4afc-8d6f-4747f3266abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:06.222647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a94a63-5571-4afc-8d6f-4747f3266abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:06.225035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a94a63-5571-4afc-8d6f-4747f3266abc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:06.248311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 234/469] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:06.248311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94a63-5571-4afc-8d6f-4747f3266abc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:06.254420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b75f68-d2aa-4c91-a539-5b677fbbac6a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:06.282233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:06.294770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986d077-cd1e-43d3-9d88-8bc6a978eefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:06.295563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986d077-cd1e-43d3-9d88-8bc6a978eefa None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.295766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986d077-cd1e-43d3-9d88-8bc6a978eefa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.296049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986d077-cd1e-43d3-9d88-8bc6a978eefa None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.296762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986d077-cd1e-43d3-9d88-8bc6a978eefa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:06.298220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986d077-cd1e-43d3-9d88-8bc6a978eefa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:06.322011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 236/470] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:01:06 2022] POST /identity/v3/users => generated 248 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:06.322841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986d077-cd1e-43d3-9d88-8bc6a978eefa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:06.401586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94a63-5571-4afc-8d6f-4747f3266abc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:06.447131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:06.451665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986d077-cd1e-43d3-9d88-8bc6a978eefa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:06.462105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323cf2a-a1b9-4921-ab7f-99320246a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:06.464952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323cf2a-a1b9-4921-ab7f-99320246a40a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.465774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323cf2a-a1b9-4921-ab7f-99320246a40a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.467517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323cf2a-a1b9-4921-ab7f-99320246a40a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.497009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb174b-b985-44cc-9717-bfd6e2d0837e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:06.497622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323cf2a-a1b9-4921-ab7f-99320246a40a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:06.497729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb174b-b985-44cc-9717-bfd6e2d0837e None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.497987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb174b-b985-44cc-9717-bfd6e2d0837e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.498270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb174b-b985-44cc-9717-bfd6e2d0837e None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.498994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb174b-b985-44cc-9717-bfd6e2d0837e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:06.500291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323cf2a-a1b9-4921-ab7f-99320246a40a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:06.501168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb174b-b985-44cc-9717-bfd6e2d0837e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:06.531316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 237/471] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:01:06 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:06.531316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb174b-b985-44cc-9717-bfd6e2d0837e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1323cf2a-a1b9-4921-ab7f-99320246a40a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:06.536687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:06.537632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:06.538486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:06.538486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:06.538486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:06.538486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:06.539655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 235/472] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:06.550364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323cf2a-a1b9-4921-ab7f-99320246a40a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:06.686733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:06.707681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb174b-b985-44cc-9717-bfd6e2d0837e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:06.720416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323cf2a-a1b9-4921-ab7f-99320246a40a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:06.748474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3596834-dec8-4d8e-a9e8-4330fffe2f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:06.749181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3596834-dec8-4d8e-a9e8-4330fffe2f8a None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.749481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3596834-dec8-4d8e-a9e8-4330fffe2f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.750661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3596834-dec8-4d8e-a9e8-4330fffe2f8a None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.751791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3596834-dec8-4d8e-a9e8-4330fffe2f8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:06.754949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3596834-dec8-4d8e-a9e8-4330fffe2f8a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:06.760208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3f01b-256f-440c-8c7c-e8e373143079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:06.760888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3f01b-256f-440c-8c7c-e8e373143079 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.762334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3f01b-256f-440c-8c7c-e8e373143079 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.762334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3f01b-256f-440c-8c7c-e8e373143079 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.762799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3f01b-256f-440c-8c7c-e8e373143079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:06.763105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3f01b-256f-440c-8c7c-e8e373143079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:06.765781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3f01b-256f-440c-8c7c-e8e373143079 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:06.781431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 236/473] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:06.782781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2ae7e-48b5-44bd-acf9-efc84d570985 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.783075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2ae7e-48b5-44bd-acf9-efc84d570985 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.783641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2ae7e-48b5-44bd-acf9-efc84d570985 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.785249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 237/474] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:06.787455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 238/475] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:01:06 2022] POST /identity/v3/users => generated 248 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:06.791870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2ae7e-48b5-44bd-acf9-efc84d570985 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:06.793273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32f5a0-9d6f-412f-8906-f2127fa2592e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.793863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32f5a0-9d6f-412f-8906-f2127fa2592e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.794344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32f5a0-9d6f-412f-8906-f2127fa2592e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.832693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0d32f5a0-9d6f-412f-8906-f2127fa2592e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:06.834171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0d32f5a0-9d6f-412f-8906-f2127fa2592e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:06.851683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:06.928995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 239/476] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:01:06.933619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2ae7e-48b5-44bd-acf9-efc84d570985 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:06.955152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32f5a0-9d6f-412f-8906-f2127fa2592e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:06.973544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb269ce-072b-4e1f-9469-3240f33312a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:06.974239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb269ce-072b-4e1f-9469-3240f33312a1 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:06.974520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb269ce-072b-4e1f-9469-3240f33312a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:06.974803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb269ce-072b-4e1f-9469-3240f33312a1 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:06.975574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb269ce-072b-4e1f-9469-3240f33312a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:06.977063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb269ce-072b-4e1f-9469-3240f33312a1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:06.989564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:07.004052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 238/477] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:01:06 2022] POST /identity/v3/users => generated 249 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:07.133671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32f5a0-9d6f-412f-8906-f2127fa2592e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:07.184266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:07.184952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:07.185240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:07.185527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:07.186292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:07.187756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:07.199601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.sql.core [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 22 11:01:07.199601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 22 11:01:07.199601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [parameters: {'id': '5a93982e4a084bcc98a9cde68a1693a7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 22 11:01:07.199601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80393) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result = self._query(query) Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application conn.query(q) Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result.read() Jul 22 11:01:07.217185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return ref.to_dict() Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 22 11:01:07.218178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application next(self.gen) Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application yield resource Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application next(self.gen) Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.session.rollback() Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.force_reraise() Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise self.value Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application session.commit() Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 22 11:01:07.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._prepare_impl() Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.session.flush() Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._flush(objects) Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application compat.raise_( Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application flush_context.execute() Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rec.execute(self) Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 22 11:01:07.220216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 22 11:01:07.221149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result = self._query(query) Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application conn.query(q) Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result.read() Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 22 11:01:07.227462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [parameters: {'id': '5a93982e4a084bcc98a9cde68a1693a7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:07.228454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 22 11:01:07.230616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:07.233554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 240/478] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 22 11:01:06 2022] POST /identity/v3/roles => generated 144 bytes in 267 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:07.237242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb269ce-072b-4e1f-9469-3240f33312a1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:07.358977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb269ce-072b-4e1f-9469-3240f33312a1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:07.396565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683aeea-91e7-4203-84f6-ffbffdacf840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:07.397392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683aeea-91e7-4203-84f6-ffbffdacf840 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:07.397683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683aeea-91e7-4203-84f6-ffbffdacf840 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:07.397965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683aeea-91e7-4203-84f6-ffbffdacf840 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:07.409822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683aeea-91e7-4203-84f6-ffbffdacf840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8683aeea-91e7-4203-84f6-ffbffdacf840 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:07.418601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:07.419518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 22 11:01:07.420341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:07.424556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 239/479] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:07.425928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:07.436653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:07.542975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f14af-fce7-4364-9a86-c8217a848f42 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:07.585661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326dc31-c2b4-423c-8e3a-0b01273ff75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:07.586344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326dc31-c2b4-423c-8e3a-0b01273ff75e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:07.586638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326dc31-c2b4-423c-8e3a-0b01273ff75e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:07.586925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326dc31-c2b4-423c-8e3a-0b01273ff75e None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:07.587721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326dc31-c2b4-423c-8e3a-0b01273ff75e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:07.588041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326dc31-c2b4-423c-8e3a-0b01273ff75e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:07.589975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326dc31-c2b4-423c-8e3a-0b01273ff75e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:07.601762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 241/480] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:09.029932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59513743-4f23-45a1-b278-08ed33fc0655 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:09.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59513743-4f23-45a1-b278-08ed33fc0655 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:09.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59513743-4f23-45a1-b278-08ed33fc0655 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:09.032081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 240/481] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:09.039230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fae84-4ecc-4e5e-894e-1a82c773f2df None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:09.039230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fae84-4ecc-4e5e-894e-1a82c773f2df None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:09.039230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fae84-4ecc-4e5e-894e-1a82c773f2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:09.065474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-376fae84-4ecc-4e5e-894e-1a82c773f2df None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:09.067015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-376fae84-4ecc-4e5e-894e-1a82c773f2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:09.181322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:09.194047 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 242/482] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:09.260193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59513743-4f23-45a1-b278-08ed33fc0655 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:09.337557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:09.399861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59513743-4f23-45a1-b278-08ed33fc0655 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:09.435412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab7324-df5e-4def-b6f0-ab2fe9576340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:09.436217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab7324-df5e-4def-b6f0-ab2fe9576340 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:09.436538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab7324-df5e-4def-b6f0-ab2fe9576340 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:09.436782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab7324-df5e-4def-b6f0-ab2fe9576340 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:09.447506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab7324-df5e-4def-b6f0-ab2fe9576340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1aab7324-df5e-4def-b6f0-ab2fe9576340 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:09.455362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:09.456323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:09.457589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 241/483] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:09.467938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fae84-4ecc-4e5e-894e-1a82c773f2df None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:09.470351 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf94e2d-9c12-4533-b2bd-fac06bf66dff None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:09.470646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf94e2d-9c12-4533-b2bd-fac06bf66dff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:09.470928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf94e2d-9c12-4533-b2bd-fac06bf66dff None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:09.472402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 242/484] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:01:09.483244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a09330-7193-4b9e-9d49-58b13529d429 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:09.483695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a09330-7193-4b9e-9d49-58b13529d429 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:09.483855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a09330-7193-4b9e-9d49-58b13529d429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:09.505928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-12a09330-7193-4b9e-9d49-58b13529d429 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:09.507164 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-12a09330-7193-4b9e-9d49-58b13529d429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:09.581846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 243/485] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:09.609762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a09330-7193-4b9e-9d49-58b13529d429 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:09.692022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:09.765388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fae84-4ecc-4e5e-894e-1a82c773f2df None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:09.824194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a09330-7193-4b9e-9d49-58b13529d429 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:09.843501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9ca3e-b832-420a-b07d-3a87d5887a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:09.844296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9ca3e-b832-420a-b07d-3a87d5887a7e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:09.844580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9ca3e-b832-420a-b07d-3a87d5887a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:09.844861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9ca3e-b832-420a-b07d-3a87d5887a7e None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:09.845627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9ca3e-b832-420a-b07d-3a87d5887a7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:09.845938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9ca3e-b832-420a-b07d-3a87d5887a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:09.847550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9ca3e-b832-420a-b07d-3a87d5887a7e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:09.857131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 243/486] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:09.858470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc1c8f-e806-4df7-8072-27adafdf320f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:09.858761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc1c8f-e806-4df7-8072-27adafdf320f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:09.859038 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc1c8f-e806-4df7-8072-27adafdf320f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:09.864977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 244/487] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:09.865998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc1c8f-e806-4df7-8072-27adafdf320f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:09.879972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb03d87-f5b0-488b-9bf0-5c396d099bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:09.881100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb03d87-f5b0-488b-9bf0-5c396d099bad None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:09.881396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb03d87-f5b0-488b-9bf0-5c396d099bad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:09.881965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb03d87-f5b0-488b-9bf0-5c396d099bad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:09.897164 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb03d87-f5b0-488b-9bf0-5c396d099bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4cb03d87-f5b0-488b-9bf0-5c396d099bad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:09.905552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:09.906456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:09.908031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 244/488] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:09.909642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a43960-300c-4133-adbc-a830cb9932cc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:09.909642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a43960-300c-4133-adbc-a830cb9932cc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:09.909642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a43960-300c-4133-adbc-a830cb9932cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:09.936409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-57a43960-300c-4133-adbc-a830cb9932cc None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:09.937652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-57a43960-300c-4133-adbc-a830cb9932cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:09.952148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:10.001270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc1c8f-e806-4df7-8072-27adafdf320f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.030176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 245/489] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:10.036513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a43960-300c-4133-adbc-a830cb9932cc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.047618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab6b78-8036-46d1-b306-9eebfec1c04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:10.049085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab6b78-8036-46d1-b306-9eebfec1c04b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.050118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab6b78-8036-46d1-b306-9eebfec1c04b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.050604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab6b78-8036-46d1-b306-9eebfec1c04b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.063383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab6b78-8036-46d1-b306-9eebfec1c04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:10.065684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab6b78-8036-46d1-b306-9eebfec1c04b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:10.085781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e9ab6b78-8036-46d1-b306-9eebfec1c04b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:10.092264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:10.093178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:10.094016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:10.094016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:10.094016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:10.094016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:10.094234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 245/490] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:10.095568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab6b78-8036-46d1-b306-9eebfec1c04b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.177085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a43960-300c-4133-adbc-a830cb9932cc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.220984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10c18-db56-485b-95fa-08365e72b345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:10.221663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10c18-db56-485b-95fa-08365e72b345 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.222018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10c18-db56-485b-95fa-08365e72b345 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.222227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10c18-db56-485b-95fa-08365e72b345 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.223186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10c18-db56-485b-95fa-08365e72b345 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:10.223332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10c18-db56-485b-95fa-08365e72b345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:10.224670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10c18-db56-485b-95fa-08365e72b345 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:10.226166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab6b78-8036-46d1-b306-9eebfec1c04b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.237992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 246/491] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:10.239715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb57c9-839b-43a7-b279-9eb413f7ee12 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.240021 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb57c9-839b-43a7-b279-9eb413f7ee12 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.240283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb57c9-839b-43a7-b279-9eb413f7ee12 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.241375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:10.243305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 247/492] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:10.243305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb57c9-839b-43a7-b279-9eb413f7ee12 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.266786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c537648-7b77-4019-b8fd-62a53f1787fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:10.267619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c537648-7b77-4019-b8fd-62a53f1787fa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.267908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c537648-7b77-4019-b8fd-62a53f1787fa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.268186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c537648-7b77-4019-b8fd-62a53f1787fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.278727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c537648-7b77-4019-b8fd-62a53f1787fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5c537648-7b77-4019-b8fd-62a53f1787fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:10.286452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:10.287422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:10.288231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:10.292530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 246/493] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:10.293172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c537648-7b77-4019-b8fd-62a53f1787fa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.345850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:10.357324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb57c9-839b-43a7-b279-9eb413f7ee12 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.398245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31fd71-60a6-4e5e-8f8b-aa5d8a063ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:10.398966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31fd71-60a6-4e5e-8f8b-aa5d8a063ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.399184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31fd71-60a6-4e5e-8f8b-aa5d8a063ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.399489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31fd71-60a6-4e5e-8f8b-aa5d8a063ac5 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.400216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31fd71-60a6-4e5e-8f8b-aa5d8a063ac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:10.400546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31fd71-60a6-4e5e-8f8b-aa5d8a063ac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:10.401949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31fd71-60a6-4e5e-8f8b-aa5d8a063ac5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:10.406662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c537648-7b77-4019-b8fd-62a53f1787fa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.418201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 248/494] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:10.418201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd4752-bdc2-4632-9e2c-a57ee5c2b4bb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.418201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd4752-bdc2-4632-9e2c-a57ee5c2b4bb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.418201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd4752-bdc2-4632-9e2c-a57ee5c2b4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.435316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-febd4752-bdc2-4632-9e2c-a57ee5c2b4bb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:10.436861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-febd4752-bdc2-4632-9e2c-a57ee5c2b4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:10.448985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b81d97-1c61-4572-85fb-c0f0f768419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:10.450196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b81d97-1c61-4572-85fb-c0f0f768419b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.450491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b81d97-1c61-4572-85fb-c0f0f768419b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.450775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b81d97-1c61-4572-85fb-c0f0f768419b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.462898 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b81d97-1c61-4572-85fb-c0f0f768419b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:10.464421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b81d97-1c61-4572-85fb-c0f0f768419b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-56b81d97-1c61-4572-85fb-c0f0f768419b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:10.481883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:10.482801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:10.483677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:10.483677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:10.483677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:10.483677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:10.483677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 247/495] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:10.483956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b81d97-1c61-4572-85fb-c0f0f768419b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.511451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:10.513849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 249/496] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:10.514740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd4752-bdc2-4632-9e2c-a57ee5c2b4bb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.611451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:10.621436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b81d97-1c61-4572-85fb-c0f0f768419b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.630185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd4752-bdc2-4632-9e2c-a57ee5c2b4bb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.656203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cad9a2-f72c-45ea-a861-fd5ed7e76d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:10.656882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cad9a2-f72c-45ea-a861-fd5ed7e76d52 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.657170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cad9a2-f72c-45ea-a861-fd5ed7e76d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.657453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cad9a2-f72c-45ea-a861-fd5ed7e76d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.658213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cad9a2-f72c-45ea-a861-fd5ed7e76d52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:10.658560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cad9a2-f72c-45ea-a861-fd5ed7e76d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:10.659992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cad9a2-f72c-45ea-a861-fd5ed7e76d52 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:10.666177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e5716-8b9e-4793-9205-9eec125fe06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:10.667503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5716-8b9e-4793-9205-9eec125fe06d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.667796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5716-8b9e-4793-9205-9eec125fe06d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.668074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5716-8b9e-4793-9205-9eec125fe06d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.669166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 248/497] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:10.669919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cad9a2-f72c-45ea-a861-fd5ed7e76d52 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.679711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e5716-8b9e-4793-9205-9eec125fe06d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:10.681204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e5716-8b9e-4793-9205-9eec125fe06d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f80e5716-8b9e-4793-9205-9eec125fe06d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:10.698713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:10.699717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:10.700865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 250/498] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:10.701725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e5716-8b9e-4793-9205-9eec125fe06d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.764409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:10.783018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cad9a2-f72c-45ea-a861-fd5ed7e76d52 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.810837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e5716-8b9e-4793-9205-9eec125fe06d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.830878 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cf347-081b-4e4c-a65b-cb02fd4022fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:10.831575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cf347-081b-4e4c-a65b-cb02fd4022fc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.831860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cf347-081b-4e4c-a65b-cb02fd4022fc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.832138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cf347-081b-4e4c-a65b-cb02fd4022fc None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.832866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cf347-081b-4e4c-a65b-cb02fd4022fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:10.833172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cf347-081b-4e4c-a65b-cb02fd4022fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:10.834545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cf347-081b-4e4c-a65b-cb02fd4022fc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:10.848939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00132a-c7ba-4048-827e-fc46316efd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:10.849741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00132a-c7ba-4048-827e-fc46316efd24 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.850036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00132a-c7ba-4048-827e-fc46316efd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.850317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00132a-c7ba-4048-827e-fc46316efd24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.858254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 249/499] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:10.858254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72908371-a391-44bf-b98d-fdd17eebe6d0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:10.858254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72908371-a391-44bf-b98d-fdd17eebe6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:10.858254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72908371-a391-44bf-b98d-fdd17eebe6d0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:10.859695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 250/500] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:10.860875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72908371-a391-44bf-b98d-fdd17eebe6d0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.864522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00132a-c7ba-4048-827e-fc46316efd24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-de00132a-c7ba-4048-827e-fc46316efd24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:10.873055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:10.874476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 251/501] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:10.875602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00132a-c7ba-4048-827e-fc46316efd24 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:10.877819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:10.970494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72908371-a391-44bf-b98d-fdd17eebe6d0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:10.977653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00132a-c7ba-4048-827e-fc46316efd24 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.007405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e5076-665a-408f-8c10-cd5c95856ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.008210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e5076-665a-408f-8c10-cd5c95856ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.008693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e5076-665a-408f-8c10-cd5c95856ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.008801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e5076-665a-408f-8c10-cd5c95856ac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.013146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54159a9-cd43-4e95-918e-076abf0d299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.013815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54159a9-cd43-4e95-918e-076abf0d299c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.014106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54159a9-cd43-4e95-918e-076abf0d299c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.014388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54159a9-cd43-4e95-918e-076abf0d299c None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.015237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54159a9-cd43-4e95-918e-076abf0d299c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:11.015648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54159a9-cd43-4e95-918e-076abf0d299c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.017108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54159a9-cd43-4e95-918e-076abf0d299c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.018937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e5076-665a-408f-8c10-cd5c95856ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.020884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e5076-665a-408f-8c10-cd5c95856ac6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.028893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ed7e5076-665a-408f-8c10-cd5c95856ac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:11.050782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:11.052633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:11.054623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:11.054623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:11.054623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:11.054623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:11.054623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 251/502] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:11.054623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e5076-665a-408f-8c10-cd5c95856ac6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.060648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 252/503] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 22 11:01:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:11.062033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224696f-1fd2-4d0c-bc34-4dec0bb31346 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.062321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224696f-1fd2-4d0c-bc34-4dec0bb31346 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.062612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224696f-1fd2-4d0c-bc34-4dec0bb31346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.088567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e224696f-1fd2-4d0c-bc34-4dec0bb31346 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:11.090331 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e224696f-1fd2-4d0c-bc34-4dec0bb31346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:11.148317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:11.162276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 253/504] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:11.163118 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224696f-1fd2-4d0c-bc34-4dec0bb31346 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.171661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e5076-665a-408f-8c10-cd5c95856ac6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.206572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77093091-0cd1-4013-8328-b0d5abdc7c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.207434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77093091-0cd1-4013-8328-b0d5abdc7c51 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.207712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77093091-0cd1-4013-8328-b0d5abdc7c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.207990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77093091-0cd1-4013-8328-b0d5abdc7c51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.216821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77093091-0cd1-4013-8328-b0d5abdc7c51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.218741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77093091-0cd1-4013-8328-b0d5abdc7c51 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-77093091-0cd1-4013-8328-b0d5abdc7c51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:11.236533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:11.237476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 252/505] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:11.238466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77093091-0cd1-4013-8328-b0d5abdc7c51 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.283887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224696f-1fd2-4d0c-bc34-4dec0bb31346 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.301789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:11.319715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1183c-c43d-4b02-8a2a-f4a2c3fc4d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.320412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1183c-c43d-4b02-8a2a-f4a2c3fc4d93 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.320710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1183c-c43d-4b02-8a2a-f4a2c3fc4d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.321011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1183c-c43d-4b02-8a2a-f4a2c3fc4d93 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.321799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1183c-c43d-4b02-8a2a-f4a2c3fc4d93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:11.322109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1183c-c43d-4b02-8a2a-f4a2c3fc4d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.324132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1183c-c43d-4b02-8a2a-f4a2c3fc4d93 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.333298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 254/506] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:11.334182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1183c-c43d-4b02-8a2a-f4a2c3fc4d93 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.346771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77093091-0cd1-4013-8328-b0d5abdc7c51 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.381671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232c8ce-5df5-4b37-9a96-99c8354b6797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.382348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232c8ce-5df5-4b37-9a96-99c8354b6797 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.382633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232c8ce-5df5-4b37-9a96-99c8354b6797 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.382911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232c8ce-5df5-4b37-9a96-99c8354b6797 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.383683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3232c8ce-5df5-4b37-9a96-99c8354b6797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:11.383993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3232c8ce-5df5-4b37-9a96-99c8354b6797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.385427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3232c8ce-5df5-4b37-9a96-99c8354b6797 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.394289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:11.400391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 253/507] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:11.402033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232c8ce-5df5-4b37-9a96-99c8354b6797 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.456190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1183c-c43d-4b02-8a2a-f4a2c3fc4d93 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.500304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352793e-5e8a-4a10-afc0-dc1b6c45ac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.501175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352793e-5e8a-4a10-afc0-dc1b6c45ac09 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.501606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352793e-5e8a-4a10-afc0-dc1b6c45ac09 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.502026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352793e-5e8a-4a10-afc0-dc1b6c45ac09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.506206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232c8ce-5df5-4b37-9a96-99c8354b6797 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.524339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:11.544586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8fa02-97bd-4e3c-80c7-04df42cb413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.545379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8fa02-97bd-4e3c-80c7-04df42cb413f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.545668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8fa02-97bd-4e3c-80c7-04df42cb413f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.545946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8fa02-97bd-4e3c-80c7-04df42cb413f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.556094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8fa02-97bd-4e3c-80c7-04df42cb413f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.557568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8fa02-97bd-4e3c-80c7-04df42cb413f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3ba8fa02-97bd-4e3c-80c7-04df42cb413f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:11.576268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:11.577297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:11.578322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:11.578322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:11.578322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:11.578322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:11.578575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352793e-5e8a-4a10-afc0-dc1b6c45ac09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.578575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352793e-5e8a-4a10-afc0-dc1b6c45ac09 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.582589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 254/508] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:11.583517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8fa02-97bd-4e3c-80c7-04df42cb413f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.631992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:11.681445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 255/509] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:11.682412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352793e-5e8a-4a10-afc0-dc1b6c45ac09 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.686453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8fa02-97bd-4e3c-80c7-04df42cb413f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.719760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae870f-582a-4236-a4e4-ad0aa49b223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.720430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae870f-582a-4236-a4e4-ad0aa49b223a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.720717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae870f-582a-4236-a4e4-ad0aa49b223a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.720996 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae870f-582a-4236-a4e4-ad0aa49b223a None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.721836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae870f-582a-4236-a4e4-ad0aa49b223a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:11.722143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae870f-582a-4236-a4e4-ad0aa49b223a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.723600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae870f-582a-4236-a4e4-ad0aa49b223a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.760269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 255/510] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:11.761571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae870f-582a-4236-a4e4-ad0aa49b223a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.782622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352793e-5e8a-4a10-afc0-dc1b6c45ac09 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.789175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:11.816821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032f77d-d590-4bcb-8b73-f9821086ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.817615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032f77d-d590-4bcb-8b73-f9821086ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.817898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032f77d-d590-4bcb-8b73-f9821086ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.818174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032f77d-d590-4bcb-8b73-f9821086ffa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.829234 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032f77d-d590-4bcb-8b73-f9821086ffa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.830712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032f77d-d590-4bcb-8b73-f9821086ffa9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4032f77d-d590-4bcb-8b73-f9821086ffa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:11.849159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:11.850090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:11.850931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:11.850931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:11.850931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:11.850931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:11.850931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 256/511] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:11.851512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032f77d-d590-4bcb-8b73-f9821086ffa9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.854551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:11.873929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae870f-582a-4236-a4e4-ad0aa49b223a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.914268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37247d00-99b3-45c9-a133-4f1c82d67809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.915111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37247d00-99b3-45c9-a133-4f1c82d67809 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.915397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37247d00-99b3-45c9-a133-4f1c82d67809 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.915676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37247d00-99b3-45c9-a133-4f1c82d67809 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.923160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37247d00-99b3-45c9-a133-4f1c82d67809 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:11.924663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37247d00-99b3-45c9-a133-4f1c82d67809 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-37247d00-99b3-45c9-a133-4f1c82d67809 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:11.940871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:11.941852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:11.942759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 256/512] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:11.943349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37247d00-99b3-45c9-a133-4f1c82d67809 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:11.963201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032f77d-d590-4bcb-8b73-f9821086ffa9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:11.997210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e8bca-ae88-441a-94a1-54c062a4734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:11.997866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e8bca-ae88-441a-94a1-54c062a4734c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:11.998149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e8bca-ae88-441a-94a1-54c062a4734c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:11.998426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e8bca-ae88-441a-94a1-54c062a4734c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:11.999144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e8bca-ae88-441a-94a1-54c062a4734c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:11.999482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e8bca-ae88-441a-94a1-54c062a4734c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.000876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e8bca-ae88-441a-94a1-54c062a4734c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.012949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 257/513] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:12.013673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e8bca-ae88-441a-94a1-54c062a4734c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:12.035850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:12.048701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37247d00-99b3-45c9-a133-4f1c82d67809 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.076157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:12.082423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee5e45-cb07-4bc9-be64-ea8d92eec671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.083224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee5e45-cb07-4bc9-be64-ea8d92eec671 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.083546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee5e45-cb07-4bc9-be64-ea8d92eec671 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.083829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee5e45-cb07-4bc9-be64-ea8d92eec671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.094361 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee5e45-cb07-4bc9-be64-ea8d92eec671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.095861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee5e45-cb07-4bc9-be64-ea8d92eec671 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-08ee5e45-cb07-4bc9-be64-ea8d92eec671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:12.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:12.114356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:12.115388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:12.115388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:12.115388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:12.115388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:12.115718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e8bca-ae88-441a-94a1-54c062a4734c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.119061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 257/514] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:12.119902 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee5e45-cb07-4bc9-be64-ea8d92eec671 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:12.150410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa61600-78cb-42fd-8a47-4578648db11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.151057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa61600-78cb-42fd-8a47-4578648db11a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.151403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa61600-78cb-42fd-8a47-4578648db11a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.151685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa61600-78cb-42fd-8a47-4578648db11a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.225361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa61600-78cb-42fd-8a47-4578648db11a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.225754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee5e45-cb07-4bc9-be64-ea8d92eec671 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.226851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa61600-78cb-42fd-8a47-4578648db11a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.259491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c90997-d789-42b0-b99e-aade5aab0054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.260148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c90997-d789-42b0-b99e-aade5aab0054 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.260430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c90997-d789-42b0-b99e-aade5aab0054 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.260711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c90997-d789-42b0-b99e-aade5aab0054 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.261431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c90997-d789-42b0-b99e-aade5aab0054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:12.261740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c90997-d789-42b0-b99e-aade5aab0054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.263166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c90997-d789-42b0-b99e-aade5aab0054 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.271632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:12.279536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 258/515] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:12.279536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c90997-d789-42b0-b99e-aade5aab0054 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:12.288081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:12.327574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 258/516] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:12.328410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa61600-78cb-42fd-8a47-4578648db11a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:12.381471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c90997-d789-42b0-b99e-aade5aab0054 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.415074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe264a3-a489-4a30-a835-33bd6e8f2328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.415779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe264a3-a489-4a30-a835-33bd6e8f2328 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.416067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe264a3-a489-4a30-a835-33bd6e8f2328 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.416343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe264a3-a489-4a30-a835-33bd6e8f2328 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.417676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe264a3-a489-4a30-a835-33bd6e8f2328 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:12.417987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe264a3-a489-4a30-a835-33bd6e8f2328 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.419450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe264a3-a489-4a30-a835-33bd6e8f2328 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.427048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa61600-78cb-42fd-8a47-4578648db11a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.436991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 259/517] 10.222.0.20 () {56 vars in 1139 bytes} [Fri Jul 22 11:01:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:12.437851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe264a3-a489-4a30-a835-33bd6e8f2328 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:12.465242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef56bb-5edf-4c39-812f-3bba7e734943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.465954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef56bb-5edf-4c39-812f-3bba7e734943 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.466240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef56bb-5edf-4c39-812f-3bba7e734943 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.466627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef56bb-5edf-4c39-812f-3bba7e734943 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.467235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef56bb-5edf-4c39-812f-3bba7e734943 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.468832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef56bb-5edf-4c39-812f-3bba7e734943 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.490058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 259/518] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:01:12 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:12.490880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef56bb-5edf-4c39-812f-3bba7e734943 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:12.501465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:12.517090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:12.557216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe264a3-a489-4a30-a835-33bd6e8f2328 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.603200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80808c-8f39-4c0c-a21b-76688606357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.603934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80808c-8f39-4c0c-a21b-76688606357a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.604221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80808c-8f39-4c0c-a21b-76688606357a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.605074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80808c-8f39-4c0c-a21b-76688606357a None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.605876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80808c-8f39-4c0c-a21b-76688606357a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:12.606184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80808c-8f39-4c0c-a21b-76688606357a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.607681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80808c-8f39-4c0c-a21b-76688606357a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.617765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef56bb-5edf-4c39-812f-3bba7e734943 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.626808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 260/519] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:12.626808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80808c-8f39-4c0c-a21b-76688606357a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:12.659595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411bf1b-b82f-4b27-8331-1e9590eb126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.660447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411bf1b-b82f-4b27-8331-1e9590eb126d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.661120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411bf1b-b82f-4b27-8331-1e9590eb126d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.661120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411bf1b-b82f-4b27-8331-1e9590eb126d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.670718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411bf1b-b82f-4b27-8331-1e9590eb126d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.672236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411bf1b-b82f-4b27-8331-1e9590eb126d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5411bf1b-b82f-4b27-8331-1e9590eb126d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:12.687355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:12.688476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 22 11:01:12.689494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:12.691354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 260/520] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:12.692248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411bf1b-b82f-4b27-8331-1e9590eb126d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:12.742484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80808c-8f39-4c0c-a21b-76688606357a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.760865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:12.781657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe465c-2742-4a77-9ca7-33d4268d9ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.781896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe465c-2742-4a77-9ca7-33d4268d9ebb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.782215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe465c-2742-4a77-9ca7-33d4268d9ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.782499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe465c-2742-4a77-9ca7-33d4268d9ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.793681 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:12.795903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe465c-2742-4a77-9ca7-33d4268d9ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.797425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe465c-2742-4a77-9ca7-33d4268d9ebb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.813474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411bf1b-b82f-4b27-8331-1e9590eb126d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3cbe465c-2742-4a77-9ca7-33d4268d9ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:12.816370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:12.817387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:12.818245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:12.818245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:12.818245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:12.818245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:12.818245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 261/521] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:12.818515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe465c-2742-4a77-9ca7-33d4268d9ebb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:12.856128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5187771-4f3a-40a6-aefa-228490b8c1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.857151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5187771-4f3a-40a6-aefa-228490b8c1cc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.857151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5187771-4f3a-40a6-aefa-228490b8c1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.857358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5187771-4f3a-40a6-aefa-228490b8c1cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.925297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe465c-2742-4a77-9ca7-33d4268d9ebb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:12.932028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5187771-4f3a-40a6-aefa-228490b8c1cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.933771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5187771-4f3a-40a6-aefa-228490b8c1cc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.960421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8212e4b-51e2-434a-aba0-f5beb62a5526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:12.961228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8212e4b-51e2-434a-aba0-f5beb62a5526 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:12.961523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8212e4b-51e2-434a-aba0-f5beb62a5526 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:12.961811 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8212e4b-51e2-434a-aba0-f5beb62a5526 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:12.969899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8212e4b-51e2-434a-aba0-f5beb62a5526 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:12.971390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8212e4b-51e2-434a-aba0-f5beb62a5526 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f8212e4b-51e2-434a-aba0-f5beb62a5526 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:12.987935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:12.989244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:12.990521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:12.990521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:12.990521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:12.990521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:12.990521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:12.990521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:12.990521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:12.990521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:12.990521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 262/522] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:12.991015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8212e4b-51e2-434a-aba0-f5beb62a5526 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.024871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:13.027051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:13.056277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 261/523] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:13.056277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5187771-4f3a-40a6-aefa-228490b8c1cc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.106340 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8212e4b-51e2-434a-aba0-f5beb62a5526 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:13.144068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf528740-3b9c-4abe-8daa-c282b6d7f49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:13.144754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf528740-3b9c-4abe-8daa-c282b6d7f49a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:13.145049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf528740-3b9c-4abe-8daa-c282b6d7f49a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:13.145330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf528740-3b9c-4abe-8daa-c282b6d7f49a None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:13.146176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf528740-3b9c-4abe-8daa-c282b6d7f49a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:13.146724 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf528740-3b9c-4abe-8daa-c282b6d7f49a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:13.147956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf528740-3b9c-4abe-8daa-c282b6d7f49a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:13.163453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5187771-4f3a-40a6-aefa-228490b8c1cc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:13.171306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 263/524] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 22 11:01:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:13.171306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf528740-3b9c-4abe-8daa-c282b6d7f49a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.206508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98aa218-8d03-471e-93e8-2863e998afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:13.207191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98aa218-8d03-471e-93e8-2863e998afb4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:13.207517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98aa218-8d03-471e-93e8-2863e998afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:13.207827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98aa218-8d03-471e-93e8-2863e998afb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:13.208663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98aa218-8d03-471e-93e8-2863e998afb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:13.209682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98aa218-8d03-471e-93e8-2863e998afb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:13.211875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98aa218-8d03-471e-93e8-2863e998afb4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:13.226813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 262/525] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:13.227965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98aa218-8d03-471e-93e8-2863e998afb4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.270832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:13.275314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:13.284182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf528740-3b9c-4abe-8daa-c282b6d7f49a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:13.320110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562c335-82bd-43cc-8fe4-ba4330f6f926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:13.320696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562c335-82bd-43cc-8fe4-ba4330f6f926 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:13.320986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562c335-82bd-43cc-8fe4-ba4330f6f926 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:13.321271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562c335-82bd-43cc-8fe4-ba4330f6f926 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:13.322067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562c335-82bd-43cc-8fe4-ba4330f6f926 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:13.322380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562c335-82bd-43cc-8fe4-ba4330f6f926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:13.324474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562c335-82bd-43cc-8fe4-ba4330f6f926 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:13.346956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 264/526] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:13.346956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562c335-82bd-43cc-8fe4-ba4330f6f926 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.349786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98aa218-8d03-471e-93e8-2863e998afb4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:13.386492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8c8de-9fa7-4ae4-8fb4-d7bfc172197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:13.387387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8c8de-9fa7-4ae4-8fb4-d7bfc172197e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:13.387682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8c8de-9fa7-4ae4-8fb4-d7bfc172197e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:13.387963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8c8de-9fa7-4ae4-8fb4-d7bfc172197e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:13.398406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8c8de-9fa7-4ae4-8fb4-d7bfc172197e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:13.399916 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8c8de-9fa7-4ae4-8fb4-d7bfc172197e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-41b8c8de-9fa7-4ae4-8fb4-d7bfc172197e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:13.417117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:13.418231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:13.419098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:13.419098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:13.419098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:13.419098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:13.419098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 263/527] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:13.419936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8c8de-9fa7-4ae4-8fb4-d7bfc172197e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.455632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562c335-82bd-43cc-8fe4-ba4330f6f926 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:13.493161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f919ce-9b79-4a3b-9892-0e540ead27ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:13.494068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f919ce-9b79-4a3b-9892-0e540ead27ef None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:13.494540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f919ce-9b79-4a3b-9892-0e540ead27ef None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:13.495546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f919ce-9b79-4a3b-9892-0e540ead27ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:13.517583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:13.523574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:13.534009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8c8de-9fa7-4ae4-8fb4-d7bfc172197e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:13.571287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeef6c-e3b1-42b5-9052-8cafd8db0a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:13.572049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeef6c-e3b1-42b5-9052-8cafd8db0a73 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:13.572314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeef6c-e3b1-42b5-9052-8cafd8db0a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:13.572590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeef6c-e3b1-42b5-9052-8cafd8db0a73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:13.573071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f919ce-9b79-4a3b-9892-0e540ead27ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:13.574534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f919ce-9b79-4a3b-9892-0e540ead27ef None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:13.581304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeef6c-e3b1-42b5-9052-8cafd8db0a73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:13.582841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeef6c-e3b1-42b5-9052-8cafd8db0a73 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-30aeef6c-e3b1-42b5-9052-8cafd8db0a73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:13.598798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:13.599636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:13.600467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:13.600467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:13.600467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:13.600467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:13.600467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:13.600467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:13.600467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:13.600467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:13.605168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 264/528] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:13.605972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeef6c-e3b1-42b5-9052-8cafd8db0a73 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.697326 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 265/529] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:13.698195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f919ce-9b79-4a3b-9892-0e540ead27ef None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.727618 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeef6c-e3b1-42b5-9052-8cafd8db0a73 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:13.752026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:13.765447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6392f1-c908-44d7-b6a3-c70d6ca56f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:13.766035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6392f1-c908-44d7-b6a3-c70d6ca56f3f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:13.766325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6392f1-c908-44d7-b6a3-c70d6ca56f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:13.766786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6392f1-c908-44d7-b6a3-c70d6ca56f3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:13.767473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6392f1-c908-44d7-b6a3-c70d6ca56f3f None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=92fc25f9ad7c49c48a909d2d92ea0a66) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:13.767794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6392f1-c908-44d7-b6a3-c70d6ca56f3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:13.769254 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6392f1-c908-44d7-b6a3-c70d6ca56f3f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:13.783297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 265/530] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/role_assignments?user.id=92fc25f9ad7c49c48a909d2d92ea0a66&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:13.783297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6392f1-c908-44d7-b6a3-c70d6ca56f3f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.812943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:13.816898 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f919ce-9b79-4a3b-9892-0e540ead27ef None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:13.871311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a4890-5f5f-4973-a0fa-6ea946ea71fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:13.871922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a4890-5f5f-4973-a0fa-6ea946ea71fc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:13.872476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a4890-5f5f-4973-a0fa-6ea946ea71fc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:13.872762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a4890-5f5f-4973-a0fa-6ea946ea71fc None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:13.874224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a4890-5f5f-4973-a0fa-6ea946ea71fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:13.874545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a4890-5f5f-4973-a0fa-6ea946ea71fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:13.875969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a4890-5f5f-4973-a0fa-6ea946ea71fc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:13.893355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 266/531] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:13.895006 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a4890-5f5f-4973-a0fa-6ea946ea71fc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:13.915132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6392f1-c908-44d7-b6a3-c70d6ca56f3f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:13.959639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a97ad-4019-4915-87ad-fed863f913f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:13.960639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a97ad-4019-4915-87ad-fed863f913f5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:13.961621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a97ad-4019-4915-87ad-fed863f913f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:13.962112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a97ad-4019-4915-87ad-fed863f913f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:13.963567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a97ad-4019-4915-87ad-fed863f913f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:13.964593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a97ad-4019-4915-87ad-fed863f913f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:13.966881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a97ad-4019-4915-87ad-fed863f913f5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:13.993023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 266/532] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:13.993712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a97ad-4019-4915-87ad-fed863f913f5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:14.051149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a4890-5f5f-4973-a0fa-6ea946ea71fc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:14.064337 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:14.085676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:14.096391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4d53a-7a8b-4e8c-bf9c-e4dfc7ccf825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:14.097510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4d53a-7a8b-4e8c-bf9c-e4dfc7ccf825 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.098034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4d53a-7a8b-4e8c-bf9c-e4dfc7ccf825 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.098594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4d53a-7a8b-4e8c-bf9c-e4dfc7ccf825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.111616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4d53a-7a8b-4e8c-bf9c-e4dfc7ccf825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:14.112212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a97ad-4019-4915-87ad-fed863f913f5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:14.114382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4d53a-7a8b-4e8c-bf9c-e4dfc7ccf825 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-89e4d53a-7a8b-4e8c-bf9c-e4dfc7ccf825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:14.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:14.135399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:14.136869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:14.136869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:14.136869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:14.136869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:14.139028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 267/533] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:14.140312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4d53a-7a8b-4e8c-bf9c-e4dfc7ccf825 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:14.154938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4556100-6b57-4f12-846d-ee99c76e87be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:14.155505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4556100-6b57-4f12-846d-ee99c76e87be None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.155796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4556100-6b57-4f12-846d-ee99c76e87be None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.156074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4556100-6b57-4f12-846d-ee99c76e87be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.165864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4556100-6b57-4f12-846d-ee99c76e87be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:14.167959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4556100-6b57-4f12-846d-ee99c76e87be None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e4556100-6b57-4f12-846d-ee99c76e87be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:14.182740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:14.183690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:14.184537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:14.184537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:14.184537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:14.184537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:14.184537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:14.184537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:14.184537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:14.184537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:14.189147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 267/534] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:14.190061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4556100-6b57-4f12-846d-ee99c76e87be None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:14.264074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4d53a-7a8b-4e8c-bf9c-e4dfc7ccf825 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:14.300541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4556100-6b57-4f12-846d-ee99c76e87be None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:14.302912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869a6ba-f0f0-48a7-a6af-21c308c8243c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:14.303855 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a6ba-f0f0-48a7-a6af-21c308c8243c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.304354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a6ba-f0f0-48a7-a6af-21c308c8243c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.304829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a6ba-f0f0-48a7-a6af-21c308c8243c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.305884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869a6ba-f0f0-48a7-a6af-21c308c8243c None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=c51929d0ecb047b4bdc405dc86fe7eb6) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:14.306396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869a6ba-f0f0-48a7-a6af-21c308c8243c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:14.308272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869a6ba-f0f0-48a7-a6af-21c308c8243c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:14.315200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 268/535] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:14 2022] GET /identity/v3/role_assignments?user.id=c51929d0ecb047b4bdc405dc86fe7eb6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:14.316311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869a6ba-f0f0-48a7-a6af-21c308c8243c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:14.338145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421e305-337e-407c-9c1a-ed986898f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:14.339427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421e305-337e-407c-9c1a-ed986898f0dd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.339782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421e305-337e-407c-9c1a-ed986898f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.340068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421e305-337e-407c-9c1a-ed986898f0dd None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.340886 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421e305-337e-407c-9c1a-ed986898f0dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:14.341209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421e305-337e-407c-9c1a-ed986898f0dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:14.342659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421e305-337e-407c-9c1a-ed986898f0dd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:14.345565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:14.351968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:14.357443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 268/536] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:14.363392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421e305-337e-407c-9c1a-ed986898f0dd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:14.459595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869a6ba-f0f0-48a7-a6af-21c308c8243c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:14.474670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421e305-337e-407c-9c1a-ed986898f0dd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:14.513672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5662-4f99-442d-bb43-29473730af4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:14.514496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5662-4f99-442d-bb43-29473730af4a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.514786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5662-4f99-442d-bb43-29473730af4a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.515059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5662-4f99-442d-bb43-29473730af4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.522904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a5662-4f99-442d-bb43-29473730af4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:14.527572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a5662-4f99-442d-bb43-29473730af4a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:14.527784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37735560-b6b8-459d-a134-0b10923783e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:14.531281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37735560-b6b8-459d-a134-0b10923783e3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.531281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37735560-b6b8-459d-a134-0b10923783e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.531281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37735560-b6b8-459d-a134-0b10923783e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.531281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37735560-b6b8-459d-a134-0b10923783e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:14.531281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37735560-b6b8-459d-a134-0b10923783e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:14.531660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37735560-b6b8-459d-a134-0b10923783e3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-787a5662-4f99-442d-bb43-29473730af4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:14.550297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:14.551203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:14.552078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:14.552078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:14.552078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:14.552078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:14.552078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:14.552078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:14.552078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:14.552078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:14.552489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 269/537] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:14.552565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 269/538] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:14.552642 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37735560-b6b8-459d-a134-0b10923783e3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:14.560442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5662-4f99-442d-bb43-29473730af4a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:14.635641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:14.681663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:14.693065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5662-4f99-442d-bb43-29473730af4a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:14.719838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37735560-b6b8-459d-a134-0b10923783e3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:14.735541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b297c4-8d87-4c0b-8cc7-7d0506647171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:14.736576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b297c4-8d87-4c0b-8cc7-7d0506647171 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.737209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b297c4-8d87-4c0b-8cc7-7d0506647171 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.737698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b297c4-8d87-4c0b-8cc7-7d0506647171 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.738705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b297c4-8d87-4c0b-8cc7-7d0506647171 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:14.739500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b297c4-8d87-4c0b-8cc7-7d0506647171 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:14.741367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b297c4-8d87-4c0b-8cc7-7d0506647171 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:14.761565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 270/539] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:14.763763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8481bb-b06b-4710-964a-d6a873634c87 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.764288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8481bb-b06b-4710-964a-d6a873634c87 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.764798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8481bb-b06b-4710-964a-d6a873634c87 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.766361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 271/540] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:14.767297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8481bb-b06b-4710-964a-d6a873634c87 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:14.803250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07278321-77b7-4d96-b97d-61319fa45680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:14.804541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07278321-77b7-4d96-b97d-61319fa45680 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.805311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07278321-77b7-4d96-b97d-61319fa45680 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.805813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07278321-77b7-4d96-b97d-61319fa45680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.808313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07278321-77b7-4d96-b97d-61319fa45680 None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=0a0ce1804b514b42845ab33c7bfee309) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:14.808880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07278321-77b7-4d96-b97d-61319fa45680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:14.811969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07278321-77b7-4d96-b97d-61319fa45680 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:14.831291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 270/541] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:14 2022] GET /identity/v3/role_assignments?user.id=0a0ce1804b514b42845ab33c7bfee309&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:14.831291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7388a7-ede7-407a-8786-0a011b838da5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:14.831291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7388a7-ede7-407a-8786-0a011b838da5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:14.831291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7388a7-ede7-407a-8786-0a011b838da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:14.869507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ea7388a7-ede7-407a-8786-0a011b838da5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:14.870757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ea7388a7-ede7-407a-8786-0a011b838da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:14.970033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8481bb-b06b-4710-964a-d6a873634c87 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:14.995982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:15.027865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 271/542] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:15.051586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7388a7-ede7-407a-8786-0a011b838da5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:15.054501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098328c-4773-45de-909a-3c6060913156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:15.055181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098328c-4773-45de-909a-3c6060913156 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:15.055492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098328c-4773-45de-909a-3c6060913156 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:15.055774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098328c-4773-45de-909a-3c6060913156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:15.056626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098328c-4773-45de-909a-3c6060913156 None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=5b04cb4117974f8eb9bbf3a41c48a11a) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:15.056949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098328c-4773-45de-909a-3c6060913156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:15.058474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098328c-4773-45de-909a-3c6060913156 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:15.076468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 272/543] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:14 2022] GET /identity/v3/role_assignments?user.id=5b04cb4117974f8eb9bbf3a41c48a11a&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:15.162429 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:15.202268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7388a7-ede7-407a-8786-0a011b838da5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:15.253312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdd3b1-1633-43ed-8e27-a3cd89820105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:15.254472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdd3b1-1633-43ed-8e27-a3cd89820105 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:15.255030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdd3b1-1633-43ed-8e27-a3cd89820105 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:15.255553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdd3b1-1633-43ed-8e27-a3cd89820105 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:15.270513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdd3b1-1633-43ed-8e27-a3cd89820105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-aabdd3b1-1633-43ed-8e27-a3cd89820105 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:15.279840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:15.280827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:15.281616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:15.284549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 272/544] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:15.292369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098328c-4773-45de-909a-3c6060913156 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:15.411232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098328c-4773-45de-909a-3c6060913156 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:15.447892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1968c-5fc5-4540-8ffb-97b9fa1fb1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:15.449581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1968c-5fc5-4540-8ffb-97b9fa1fb1ad None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:15.450099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1968c-5fc5-4540-8ffb-97b9fa1fb1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:15.450582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1968c-5fc5-4540-8ffb-97b9fa1fb1ad None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:15.451592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1968c-5fc5-4540-8ffb-97b9fa1fb1ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:15.452128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1968c-5fc5-4540-8ffb-97b9fa1fb1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:15.453714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1968c-5fc5-4540-8ffb-97b9fa1fb1ad None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:15.463425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 273/545] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:15.473061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdd3b1-1633-43ed-8e27-a3cd89820105 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:15.699151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdd3b1-1633-43ed-8e27-a3cd89820105 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:15.750624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65968b-8ad4-4ee0-a380-e3b838d1e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:15.751463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65968b-8ad4-4ee0-a380-e3b838d1e452 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:15.751756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65968b-8ad4-4ee0-a380-e3b838d1e452 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:15.752033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65968b-8ad4-4ee0-a380-e3b838d1e452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:15.759477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:15.777180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65968b-8ad4-4ee0-a380-e3b838d1e452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:15.777180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65968b-8ad4-4ee0-a380-e3b838d1e452 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ec65968b-8ad4-4ee0-a380-e3b838d1e452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:15.803637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:15.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:15.805410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:15.805410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:15.805410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:15.805410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:15.812504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 273/546] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:15.814813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1968c-5fc5-4540-8ffb-97b9fa1fb1ad None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:15.929126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:16.053121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1968c-5fc5-4540-8ffb-97b9fa1fb1ad None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:16.120889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bf0e-29a2-4832-b124-fef53f5f21b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:16.122156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bf0e-29a2-4832-b124-fef53f5f21b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:16.122656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bf0e-29a2-4832-b124-fef53f5f21b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:16.123150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bf0e-29a2-4832-b124-fef53f5f21b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:16.124541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289bf0e-29a2-4832-b124-fef53f5f21b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:16.125100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289bf0e-29a2-4832-b124-fef53f5f21b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:16.127690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289bf0e-29a2-4832-b124-fef53f5f21b2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:16.153568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 274/547] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:16.159125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65968b-8ad4-4ee0-a380-e3b838d1e452 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:16.341952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65968b-8ad4-4ee0-a380-e3b838d1e452 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:16.385407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49232f-b86c-44c7-acbe-10e07a8e0580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:16.386222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49232f-b86c-44c7-acbe-10e07a8e0580 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:16.386508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49232f-b86c-44c7-acbe-10e07a8e0580 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:16.386773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49232f-b86c-44c7-acbe-10e07a8e0580 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:16.397362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49232f-b86c-44c7-acbe-10e07a8e0580 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:16.398823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49232f-b86c-44c7-acbe-10e07a8e0580 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-6f49232f-b86c-44c7-acbe-10e07a8e0580 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:16.415993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:16.418219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:16.422685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 274/548] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:16.429093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bf0e-29a2-4832-b124-fef53f5f21b2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:16.637745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bf0e-29a2-4832-b124-fef53f5f21b2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:16.639494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:16.671586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a7c20-18ec-492a-b19d-e8ee35e2066a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:16.671899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a7c20-18ec-492a-b19d-e8ee35e2066a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:16.672158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a7c20-18ec-492a-b19d-e8ee35e2066a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:16.677257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 275/549] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:16.678545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c3da0-ac79-46d4-9b43-fffa6efcab99 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:16.678842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c3da0-ac79-46d4-9b43-fffa6efcab99 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:16.679127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c3da0-ac79-46d4-9b43-fffa6efcab99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:16.688271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332cd83-fc7b-441f-8864-f20fe24134dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:16.688954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332cd83-fc7b-441f-8864-f20fe24134dd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:16.689244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332cd83-fc7b-441f-8864-f20fe24134dd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:16.689531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332cd83-fc7b-441f-8864-f20fe24134dd None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:16.690392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332cd83-fc7b-441f-8864-f20fe24134dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:16.690707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332cd83-fc7b-441f-8864-f20fe24134dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:16.692204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332cd83-fc7b-441f-8864-f20fe24134dd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:16.715509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 275/550] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 22 11:01:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:16.715785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1f3c3da0-ac79-46d4-9b43-fffa6efcab99 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:16.717189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1f3c3da0-ac79-46d4-9b43-fffa6efcab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:16.729941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332cd83-fc7b-441f-8864-f20fe24134dd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:16.790903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:16.814550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 276/551] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:16.837590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c3da0-ac79-46d4-9b43-fffa6efcab99 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:16.949016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c3da0-ac79-46d4-9b43-fffa6efcab99 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:16.950035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332cd83-fc7b-441f-8864-f20fe24134dd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:16.995673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df718a-6e57-4a71-b43f-cfaa05cb5e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:16.996865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df718a-6e57-4a71-b43f-cfaa05cb5e2c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:16.997377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df718a-6e57-4a71-b43f-cfaa05cb5e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:16.997828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df718a-6e57-4a71-b43f-cfaa05cb5e2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.009102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df718a-6e57-4a71-b43f-cfaa05cb5e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-75df718a-6e57-4a71-b43f-cfaa05cb5e2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:17.018601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:17.019698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:17.020611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:17.023044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 277/552] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:17.029132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df718a-6e57-4a71-b43f-cfaa05cb5e2c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:17.045065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef3ca-5414-4b74-a219-44be21c57967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:17.046740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef3ca-5414-4b74-a219-44be21c57967 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.047216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef3ca-5414-4b74-a219-44be21c57967 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.047688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef3ca-5414-4b74-a219-44be21c57967 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.089040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:17.091726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:17.159464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef3ca-5414-4b74-a219-44be21c57967 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:17.161762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef3ca-5414-4b74-a219-44be21c57967 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:17.169434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df718a-6e57-4a71-b43f-cfaa05cb5e2c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:17.216340 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b6a92-5323-40d1-baca-72fc2d287d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:17.216942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b6a92-5323-40d1-baca-72fc2d287d77 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.217232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b6a92-5323-40d1-baca-72fc2d287d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.217511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b6a92-5323-40d1-baca-72fc2d287d77 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.218270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b6a92-5323-40d1-baca-72fc2d287d77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:17.218586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b6a92-5323-40d1-baca-72fc2d287d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:17.220657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b6a92-5323-40d1-baca-72fc2d287d77 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:17.232915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 278/553] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:17.234275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c5a5a-31ab-4ad5-8c8b-c5f65a14649c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.234572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c5a5a-31ab-4ad5-8c8b-c5f65a14649c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.234856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c5a5a-31ab-4ad5-8c8b-c5f65a14649c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.241457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 279/554] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:17.242314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c5a5a-31ab-4ad5-8c8b-c5f65a14649c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:17.305466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 276/555] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:17.307871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced94d92-3c34-40f3-9bea-3fc5c435ad59 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.308374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced94d92-3c34-40f3-9bea-3fc5c435ad59 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.308951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced94d92-3c34-40f3-9bea-3fc5c435ad59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.333236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ced94d92-3c34-40f3-9bea-3fc5c435ad59 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:17.335961 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ced94d92-3c34-40f3-9bea-3fc5c435ad59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:17.383539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c5a5a-31ab-4ad5-8c8b-c5f65a14649c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:17.385628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:17.409928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:17.426500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 277/556] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:17.427956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced94d92-3c34-40f3-9bea-3fc5c435ad59 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:17.444224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164fca6-7bc8-4f7d-beaa-4085350a7003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:17.445355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164fca6-7bc8-4f7d-beaa-4085350a7003 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.445862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164fca6-7bc8-4f7d-beaa-4085350a7003 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.446333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164fca6-7bc8-4f7d-beaa-4085350a7003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.460189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164fca6-7bc8-4f7d-beaa-4085350a7003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:17.462105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164fca6-7bc8-4f7d-beaa-4085350a7003 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b164fca6-7bc8-4f7d-beaa-4085350a7003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:17.483423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:17.484341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:17.485210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:17.485210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:17.485210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:17.485210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:17.488174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 280/557] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:17.489567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164fca6-7bc8-4f7d-beaa-4085350a7003 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:17.555510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced94d92-3c34-40f3-9bea-3fc5c435ad59 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:17.594898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663625-0832-4a61-9e89-43a5590c90f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:17.595740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663625-0832-4a61-9e89-43a5590c90f1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.596027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663625-0832-4a61-9e89-43a5590c90f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.596305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663625-0832-4a61-9e89-43a5590c90f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.610626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b663625-0832-4a61-9e89-43a5590c90f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:17.612143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b663625-0832-4a61-9e89-43a5590c90f1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:17.613423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164fca6-7bc8-4f7d-beaa-4085350a7003 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1b663625-0832-4a61-9e89-43a5590c90f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:17.630996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:17.632451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:17.633356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:17.633356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:17.633356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:17.633356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:17.633356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 278/558] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:17.633356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663625-0832-4a61-9e89-43a5590c90f1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:17.659787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa52a2e-6f8b-4087-9aea-2987871670a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:17.660593 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa52a2e-6f8b-4087-9aea-2987871670a7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.660883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa52a2e-6f8b-4087-9aea-2987871670a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.661159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa52a2e-6f8b-4087-9aea-2987871670a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.675580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa52a2e-6f8b-4087-9aea-2987871670a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4aa52a2e-6f8b-4087-9aea-2987871670a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:17.684172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:17.685118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:17.686045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 281/559] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:17.687232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d68bb-f18c-4be1-966d-89d23ce82889 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.687367 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d68bb-f18c-4be1-966d-89d23ce82889 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.687591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d68bb-f18c-4be1-966d-89d23ce82889 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.693598 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 282/560] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:17.694557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d68bb-f18c-4be1-966d-89d23ce82889 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:17.696721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:17.711736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:17.757279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663625-0832-4a61-9e89-43a5590c90f1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:17.798627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a56e9b-b5ef-4172-a6d0-94493805e846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:17.799452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a56e9b-b5ef-4172-a6d0-94493805e846 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.799739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a56e9b-b5ef-4172-a6d0-94493805e846 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.800128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a56e9b-b5ef-4172-a6d0-94493805e846 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.800905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a56e9b-b5ef-4172-a6d0-94493805e846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:17.801219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a56e9b-b5ef-4172-a6d0-94493805e846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:17.802689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a56e9b-b5ef-4172-a6d0-94493805e846 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:17.807394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d68bb-f18c-4be1-966d-89d23ce82889 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:17.819282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 279/561] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:17.819282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a56e9b-b5ef-4172-a6d0-94493805e846 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:17.852922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcd6b4-070e-4ce7-a17e-3f35b73d2dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:17.853613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcd6b4-070e-4ce7-a17e-3f35b73d2dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.853907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcd6b4-070e-4ce7-a17e-3f35b73d2dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.855412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcd6b4-070e-4ce7-a17e-3f35b73d2dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.855942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcd6b4-070e-4ce7-a17e-3f35b73d2dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:17.856555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcd6b4-070e-4ce7-a17e-3f35b73d2dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:17.864083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcd6b4-070e-4ce7-a17e-3f35b73d2dc0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:17.881636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 283/562] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:17.881636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc453ad1-89cf-4c62-b3dc-fcdae0504c79 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.881636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc453ad1-89cf-4c62-b3dc-fcdae0504c79 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.881636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc453ad1-89cf-4c62-b3dc-fcdae0504c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.896232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bc453ad1-89cf-4c62-b3dc-fcdae0504c79 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:17.897452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bc453ad1-89cf-4c62-b3dc-fcdae0504c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:17.942486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a56e9b-b5ef-4172-a6d0-94493805e846 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:17.944482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:17.969326 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 284/563] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:17.970448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc453ad1-89cf-4c62-b3dc-fcdae0504c79 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:17.981988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:17.988063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4a64e-c044-4c58-8d25-3f041298ccbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:17.989119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4a64e-c044-4c58-8d25-3f041298ccbf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:17.989515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4a64e-c044-4c58-8d25-3f041298ccbf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:17.989813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4a64e-c044-4c58-8d25-3f041298ccbf None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:17.990582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4a64e-c044-4c58-8d25-3f041298ccbf None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:17.990894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4a64e-c044-4c58-8d25-3f041298ccbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:17.992310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4a64e-c044-4c58-8d25-3f041298ccbf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.001932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 280/564] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:18.002776 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4a64e-c044-4c58-8d25-3f041298ccbf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.095011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc453ad1-89cf-4c62-b3dc-fcdae0504c79 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.114181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4a64e-c044-4c58-8d25-3f041298ccbf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.128947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f96016-7cb9-493a-80a8-3d51e2a4ee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.130048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f96016-7cb9-493a-80a8-3d51e2a4ee38 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.130374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f96016-7cb9-493a-80a8-3d51e2a4ee38 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.130727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f96016-7cb9-493a-80a8-3d51e2a4ee38 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.138580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f96016-7cb9-493a-80a8-3d51e2a4ee38 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:18.140072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f96016-7cb9-493a-80a8-3d51e2a4ee38 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.149135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a775884-f126-4ac1-8db5-6ebc9755a2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.149911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a775884-f126-4ac1-8db5-6ebc9755a2be None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.150196 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a775884-f126-4ac1-8db5-6ebc9755a2be None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.150475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a775884-f126-4ac1-8db5-6ebc9755a2be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c8f96016-7cb9-493a-80a8-3d51e2a4ee38 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:18.154881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:18.156057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:18.157048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 285/565] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:18.157657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cd4fc-7e70-4d37-a0d2-9187320165e7 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.157657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cd4fc-7e70-4d37-a0d2-9187320165e7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.157856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cd4fc-7e70-4d37-a0d2-9187320165e7 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.159017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a775884-f126-4ac1-8db5-6ebc9755a2be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:18.163019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 286/566] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:18.163019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cd4fc-7e70-4d37-a0d2-9187320165e7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.163384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a775884-f126-4ac1-8db5-6ebc9755a2be None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2a775884-f126-4ac1-8db5-6ebc9755a2be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:18.177158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:18.178210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 281/567] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:18.179098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a775884-f126-4ac1-8db5-6ebc9755a2be None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.206379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:18.235827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:18.274142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cd4fc-7e70-4d37-a0d2-9187320165e7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.282301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a775884-f126-4ac1-8db5-6ebc9755a2be None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.314315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833f9d8-0259-4208-8817-d59100a31ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.315169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833f9d8-0259-4208-8817-d59100a31ace None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.315505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833f9d8-0259-4208-8817-d59100a31ace None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.315793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833f9d8-0259-4208-8817-d59100a31ace None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.319587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018ff86-7d7c-4a5c-8c05-1a698def8348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.320383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ff86-7d7c-4a5c-8c05-1a698def8348 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.320675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ff86-7d7c-4a5c-8c05-1a698def8348 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.320960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ff86-7d7c-4a5c-8c05-1a698def8348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.327065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833f9d8-0259-4208-8817-d59100a31ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:18.331726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018ff86-7d7c-4a5c-8c05-1a698def8348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:18.333191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018ff86-7d7c-4a5c-8c05-1a698def8348 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c833f9d8-0259-4208-8817-d59100a31ace None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:18.334795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:18.335844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:18.336673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 287/568] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:18.337389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833f9d8-0259-4208-8817-d59100a31ace None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c018ff86-7d7c-4a5c-8c05-1a698def8348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:18.351663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:18.352782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:18.354418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:18.354418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:18.354418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:18.354418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:18.354418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 282/569] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:18.354418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7fb14-1219-4668-a2c3-f958fc0f29e0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.354418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7fb14-1219-4668-a2c3-f958fc0f29e0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.355689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7fb14-1219-4668-a2c3-f958fc0f29e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.371551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5bd7fb14-1219-4668-a2c3-f958fc0f29e0 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:18.372769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5bd7fb14-1219-4668-a2c3-f958fc0f29e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:18.446633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 283/570] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:18.447857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7fb14-1219-4668-a2c3-f958fc0f29e0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.455822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833f9d8-0259-4208-8817-d59100a31ace None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.469904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:18.480562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:18.491689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f993-e9a3-40f2-a049-2965b194605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.492360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f993-e9a3-40f2-a049-2965b194605e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.492645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f993-e9a3-40f2-a049-2965b194605e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.492925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f993-e9a3-40f2-a049-2965b194605e None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.493757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421f993-e9a3-40f2-a049-2965b194605e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:18.494069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421f993-e9a3-40f2-a049-2965b194605e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:18.496024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421f993-e9a3-40f2-a049-2965b194605e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.513637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 288/571] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:18.514462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f993-e9a3-40f2-a049-2965b194605e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.561860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7fb14-1219-4668-a2c3-f958fc0f29e0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.603533 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f80652-b439-42db-a9bc-7826e9409a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.604248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f80652-b439-42db-a9bc-7826e9409a4e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.604533 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f80652-b439-42db-a9bc-7826e9409a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.604808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f80652-b439-42db-a9bc-7826e9409a4e None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.605685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f80652-b439-42db-a9bc-7826e9409a4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:18.605976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f80652-b439-42db-a9bc-7826e9409a4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:18.607452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f80652-b439-42db-a9bc-7826e9409a4e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.614082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f993-e9a3-40f2-a049-2965b194605e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.620353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 284/572] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:18.621273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f80652-b439-42db-a9bc-7826e9409a4e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.654201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d15adf-78d5-45af-918c-655e20bdacd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.654883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15adf-78d5-45af-918c-655e20bdacd1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.655172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15adf-78d5-45af-918c-655e20bdacd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.655481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15adf-78d5-45af-918c-655e20bdacd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.656253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d15adf-78d5-45af-918c-655e20bdacd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:18.656561 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d15adf-78d5-45af-918c-655e20bdacd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:18.657966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d15adf-78d5-45af-918c-655e20bdacd1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.667227 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 289/573] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:18.668126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d15adf-78d5-45af-918c-655e20bdacd1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.693655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:18.718104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:18.731424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f80652-b439-42db-a9bc-7826e9409a4e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.771138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80abf0a-e00a-4415-b257-00b5121309f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.771832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80abf0a-e00a-4415-b257-00b5121309f9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.772116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80abf0a-e00a-4415-b257-00b5121309f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.772390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80abf0a-e00a-4415-b257-00b5121309f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.773122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80abf0a-e00a-4415-b257-00b5121309f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:18.773428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80abf0a-e00a-4415-b257-00b5121309f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:18.774801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80abf0a-e00a-4415-b257-00b5121309f9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.776432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d15adf-78d5-45af-918c-655e20bdacd1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.787085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 285/574] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:18.787962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80abf0a-e00a-4415-b257-00b5121309f9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.812720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16df233-5acb-4af6-8eb8-b2351132cc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.813511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16df233-5acb-4af6-8eb8-b2351132cc20 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.813800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16df233-5acb-4af6-8eb8-b2351132cc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.814080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16df233-5acb-4af6-8eb8-b2351132cc20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.824162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16df233-5acb-4af6-8eb8-b2351132cc20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a16df233-5acb-4af6-8eb8-b2351132cc20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:18.831880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:18.832860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:18.833691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 290/575] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:18.834394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16df233-5acb-4af6-8eb8-b2351132cc20 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.891507 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80abf0a-e00a-4415-b257-00b5121309f9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.918801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:18.928510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07b889-b2d6-4cfe-83b7-bfac540db407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.929343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07b889-b2d6-4cfe-83b7-bfac540db407 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.929635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07b889-b2d6-4cfe-83b7-bfac540db407 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.929925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07b889-b2d6-4cfe-83b7-bfac540db407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.939088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16df233-5acb-4af6-8eb8-b2351132cc20 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:18.941391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07b889-b2d6-4cfe-83b7-bfac540db407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:18.941523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:18.942919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07b889-b2d6-4cfe-83b7-bfac540db407 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9a07b889-b2d6-4cfe-83b7-bfac540db407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:18.961030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:18.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:18.963847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:18.963847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:18.963847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:18.963847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:18.963847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 286/576] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:18.963847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07b889-b2d6-4cfe-83b7-bfac540db407 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:18.986475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e478c63-288e-411b-aa82-e810379109e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:18.987149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e478c63-288e-411b-aa82-e810379109e0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:18.987468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e478c63-288e-411b-aa82-e810379109e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:18.987749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e478c63-288e-411b-aa82-e810379109e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:18.988991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e478c63-288e-411b-aa82-e810379109e0 None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=e7935e85cc3f4ed1ad5b01ba892e6e82) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:18.989409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e478c63-288e-411b-aa82-e810379109e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:18.990608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e478c63-288e-411b-aa82-e810379109e0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:18.997257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 291/577] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/role_assignments?user.id=e7935e85cc3f4ed1ad5b01ba892e6e82&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:18.998068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e478c63-288e-411b-aa82-e810379109e0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.084086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07b889-b2d6-4cfe-83b7-bfac540db407 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.118068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e478c63-288e-411b-aa82-e810379109e0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.123276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9bb7c-bd21-45cc-8023-1b178b1be80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.124109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9bb7c-bd21-45cc-8023-1b178b1be80b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.124397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9bb7c-bd21-45cc-8023-1b178b1be80b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.124697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9bb7c-bd21-45cc-8023-1b178b1be80b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.137080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9bb7c-bd21-45cc-8023-1b178b1be80b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.138568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9bb7c-bd21-45cc-8023-1b178b1be80b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-dda9bb7c-bd21-45cc-8023-1b178b1be80b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:19.157237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:19.158167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:19.159051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:19.159051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:19.159051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:19.159051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:19.159051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 287/578] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:19.159406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9bb7c-bd21-45cc-8023-1b178b1be80b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.160866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a2e34-f289-43ae-b2f7-a24b27b40c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.161921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a2e34-f289-43ae-b2f7-a24b27b40c13 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.162447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a2e34-f289-43ae-b2f7-a24b27b40c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.162947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a2e34-f289-43ae-b2f7-a24b27b40c13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.172272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a2e34-f289-43ae-b2f7-a24b27b40c13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.173821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a2e34-f289-43ae-b2f7-a24b27b40c13 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-330a2e34-f289-43ae-b2f7-a24b27b40c13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:19.189747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:19.190696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 292/579] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 22 11:01:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:19.191863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a2e34-f289-43ae-b2f7-a24b27b40c13 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.193485 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:19.222006 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:19.273498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9bb7c-bd21-45cc-8023-1b178b1be80b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.299043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a2e34-f289-43ae-b2f7-a24b27b40c13 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.310121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bc72c-d03c-477f-b10a-a8efe92f19a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.310791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bc72c-d03c-477f-b10a-a8efe92f19a3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.311075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bc72c-d03c-477f-b10a-a8efe92f19a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.311375 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bc72c-d03c-477f-b10a-a8efe92f19a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.312136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bc72c-d03c-477f-b10a-a8efe92f19a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:19.312444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bc72c-d03c-477f-b10a-a8efe92f19a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.314082 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bc72c-d03c-477f-b10a-a8efe92f19a3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.322212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 288/580] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:19.322950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bc72c-d03c-477f-b10a-a8efe92f19a3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.339754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4792c8-2853-4549-aa0d-5bf406b29bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.340743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4792c8-2853-4549-aa0d-5bf406b29bcf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.341179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4792c8-2853-4549-aa0d-5bf406b29bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.341610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4792c8-2853-4549-aa0d-5bf406b29bcf None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.342606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4792c8-2853-4549-aa0d-5bf406b29bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:19.343074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4792c8-2853-4549-aa0d-5bf406b29bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.344665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4792c8-2853-4549-aa0d-5bf406b29bcf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.358655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 293/581] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:19.359755 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4792c8-2853-4549-aa0d-5bf406b29bcf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.432958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:19.443853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bc72c-d03c-477f-b10a-a8efe92f19a3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.481618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:19.489124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db995ff-65e6-4607-b13f-94301dba6f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.489894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db995ff-65e6-4607-b13f-94301dba6f00 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.490248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db995ff-65e6-4607-b13f-94301dba6f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.490583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db995ff-65e6-4607-b13f-94301dba6f00 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.491357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db995ff-65e6-4607-b13f-94301dba6f00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:19.491674 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db995ff-65e6-4607-b13f-94301dba6f00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.493061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db995ff-65e6-4607-b13f-94301dba6f00 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.497669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4792c8-2853-4549-aa0d-5bf406b29bcf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.506609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 289/582] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:19.508036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db995ff-65e6-4607-b13f-94301dba6f00 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.539477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db1aec-c407-4e31-95af-ff5f55fe9b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.540292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db1aec-c407-4e31-95af-ff5f55fe9b7c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.540584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db1aec-c407-4e31-95af-ff5f55fe9b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.540867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db1aec-c407-4e31-95af-ff5f55fe9b7c None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.542296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db1aec-c407-4e31-95af-ff5f55fe9b7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:19.542800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db1aec-c407-4e31-95af-ff5f55fe9b7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.545105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db1aec-c407-4e31-95af-ff5f55fe9b7c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.568792 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 294/583] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:19.568792 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db1aec-c407-4e31-95af-ff5f55fe9b7c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.649753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db995ff-65e6-4607-b13f-94301dba6f00 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.691464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832e3d9-70a1-49c9-9ab3-9e62ec728952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.691907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832e3d9-70a1-49c9-9ab3-9e62ec728952 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.692196 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832e3d9-70a1-49c9-9ab3-9e62ec728952 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.692475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832e3d9-70a1-49c9-9ab3-9e62ec728952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.696266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db1aec-c407-4e31-95af-ff5f55fe9b7c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.705058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832e3d9-70a1-49c9-9ab3-9e62ec728952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.706573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832e3d9-70a1-49c9-9ab3-9e62ec728952 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5832e3d9-70a1-49c9-9ab3-9e62ec728952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:19.726998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:19.728120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:19.729205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:19.729205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:19.729205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:19.729205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:19.729205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 290/584] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:19.729205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832e3d9-70a1-49c9-9ab3-9e62ec728952 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.737945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:19.753562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6d27b-d25a-4b68-9f37-ba0d275eecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.755227 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6d27b-d25a-4b68-9f37-ba0d275eecb3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.755885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6d27b-d25a-4b68-9f37-ba0d275eecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.756328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6d27b-d25a-4b68-9f37-ba0d275eecb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.766677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6d27b-d25a-4b68-9f37-ba0d275eecb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.769177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6d27b-d25a-4b68-9f37-ba0d275eecb3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-41c6d27b-d25a-4b68-9f37-ba0d275eecb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:19.785842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:19.786796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:19.787806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:19.787806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:19.787806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:19.787806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:19.787806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:19.787806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:19.787806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:19.787806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:19.787806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 295/585] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:19.788981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6d27b-d25a-4b68-9f37-ba0d275eecb3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.801113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:19.849439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832e3d9-70a1-49c9-9ab3-9e62ec728952 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.886032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef646f2-76e2-48d6-a409-5335b630be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.886928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef646f2-76e2-48d6-a409-5335b630be75 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.887304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef646f2-76e2-48d6-a409-5335b630be75 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.887647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef646f2-76e2-48d6-a409-5335b630be75 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.895426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef646f2-76e2-48d6-a409-5335b630be75 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.896960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef646f2-76e2-48d6-a409-5335b630be75 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.906877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6d27b-d25a-4b68-9f37-ba0d275eecb3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5ef646f2-76e2-48d6-a409-5335b630be75 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:19.912531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:19.913493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 22 11:01:19.914434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:19.918544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 291/586] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:19.919150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef646f2-76e2-48d6-a409-5335b630be75 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.946673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e465a-b84c-4af4-9829-1590dd3d5d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:19.947533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e465a-b84c-4af4-9829-1590dd3d5d16 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:19.948384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e465a-b84c-4af4-9829-1590dd3d5d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:19.948678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e465a-b84c-4af4-9829-1590dd3d5d16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:19.960745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e465a-b84c-4af4-9829-1590dd3d5d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:19.962263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e465a-b84c-4af4-9829-1590dd3d5d16 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0e0e465a-b84c-4af4-9829-1590dd3d5d16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:19.980234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:19.981383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:19.987147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:19.987147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:19.987147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:19.987147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:19.987147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 296/587] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:19.987147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e465a-b84c-4af4-9829-1590dd3d5d16 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:19.994215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:20.038633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef646f2-76e2-48d6-a409-5335b630be75 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.050766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:20.085310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63643409-21cb-4873-98a5-af2bc0ca251a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.086095 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63643409-21cb-4873-98a5-af2bc0ca251a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.086457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63643409-21cb-4873-98a5-af2bc0ca251a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.086802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63643409-21cb-4873-98a5-af2bc0ca251a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.087677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63643409-21cb-4873-98a5-af2bc0ca251a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:20.088037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63643409-21cb-4873-98a5-af2bc0ca251a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.089559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63643409-21cb-4873-98a5-af2bc0ca251a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.099183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e465a-b84c-4af4-9829-1590dd3d5d16 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.102705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 292/588] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:20.103182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63643409-21cb-4873-98a5-af2bc0ca251a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:20.143439 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5580fb-d361-4632-8398-9c879136c29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.144196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5580fb-d361-4632-8398-9c879136c29a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.144564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5580fb-d361-4632-8398-9c879136c29a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.144919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5580fb-d361-4632-8398-9c879136c29a None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.145795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5580fb-d361-4632-8398-9c879136c29a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:20.146187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5580fb-d361-4632-8398-9c879136c29a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.147736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5580fb-d361-4632-8398-9c879136c29a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.165600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 297/589] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:20.165600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5580fb-d361-4632-8398-9c879136c29a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:20.218168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63643409-21cb-4873-98a5-af2bc0ca251a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.244019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:20.256134 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a8300-d45f-43b0-8eb9-0fb7ee1ec70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.256956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a8300-d45f-43b0-8eb9-0fb7ee1ec70c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.257153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a8300-d45f-43b0-8eb9-0fb7ee1ec70c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.257435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a8300-d45f-43b0-8eb9-0fb7ee1ec70c None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.258280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a8300-d45f-43b0-8eb9-0fb7ee1ec70c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:20.258736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a8300-d45f-43b0-8eb9-0fb7ee1ec70c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.260820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a8300-d45f-43b0-8eb9-0fb7ee1ec70c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.278884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5580fb-d361-4632-8398-9c879136c29a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.279057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 293/590] 10.222.0.20 () {56 vars in 1139 bytes} [Fri Jul 22 11:01:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:20.279923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a8300-d45f-43b0-8eb9-0fb7ee1ec70c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:20.302918 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:20.321872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614cb60-17e4-4bce-ad9d-05f1779cea15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.322537 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614cb60-17e4-4bce-ad9d-05f1779cea15 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614cb60-17e4-4bce-ad9d-05f1779cea15 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.323188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614cb60-17e4-4bce-ad9d-05f1779cea15 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.323946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614cb60-17e4-4bce-ad9d-05f1779cea15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:20.324306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614cb60-17e4-4bce-ad9d-05f1779cea15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.325712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614cb60-17e4-4bce-ad9d-05f1779cea15 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.337234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 298/591] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:20.338037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614cb60-17e4-4bce-ad9d-05f1779cea15 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:20.410672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a8300-d45f-43b0-8eb9-0fb7ee1ec70c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.448317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f68d53-33c6-4f87-b0c4-8c85a1680bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.449155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f68d53-33c6-4f87-b0c4-8c85a1680bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.449455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f68d53-33c6-4f87-b0c4-8c85a1680bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.449725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f68d53-33c6-4f87-b0c4-8c85a1680bf1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.456115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614cb60-17e4-4bce-ad9d-05f1779cea15 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.457718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f68d53-33c6-4f87-b0c4-8c85a1680bf1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.459540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f68d53-33c6-4f87-b0c4-8c85a1680bf1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-18f68d53-33c6-4f87-b0c4-8c85a1680bf1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:20.475836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:20.476984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 22 11:01:20.478116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:20.482475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 294/592] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:20.483629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f68d53-33c6-4f87-b0c4-8c85a1680bf1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:20.494393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc60953-7e5c-411b-a7dd-135a06c5071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.496024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc60953-7e5c-411b-a7dd-135a06c5071b None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.496381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc60953-7e5c-411b-a7dd-135a06c5071b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.496713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc60953-7e5c-411b-a7dd-135a06c5071b None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/92fc25f9ad7c49c48a909d2d92ea0a66/roles/a9580499eef24770867aeac1602aa002` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.513196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc60953-7e5c-411b-a7dd-135a06c5071b None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=92fc25f9ad7c49c48a909d2d92ea0a66, role_id=a9580499eef24770867aeac1602aa002)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.514771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc60953-7e5c-411b-a7dd-135a06c5071b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.539341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 299/593] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:01:20 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/92fc25f9ad7c49c48a909d2d92ea0a66/roles/a9580499eef24770867aeac1602aa002 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:01:20.539341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc60953-7e5c-411b-a7dd-135a06c5071b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:20.543522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:20.569487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:20.625302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f68d53-33c6-4f87-b0c4-8c85a1680bf1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.670856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc60953-7e5c-411b-a7dd-135a06c5071b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.685074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a7a8e-e274-4264-bcbc-524131b2e17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.687478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a7a8e-e274-4264-bcbc-524131b2e17f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.688235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a7a8e-e274-4264-bcbc-524131b2e17f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.689979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a7a8e-e274-4264-bcbc-524131b2e17f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.712821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a7a8e-e274-4264-bcbc-524131b2e17f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.716718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a7a8e-e274-4264-bcbc-524131b2e17f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.724632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a947d13-1b76-471f-adec-363835960659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.724927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a947d13-1b76-471f-adec-363835960659 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.725344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a947d13-1b76-471f-adec-363835960659 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.725728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a947d13-1b76-471f-adec-363835960659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.735466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a947d13-1b76-471f-adec-363835960659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.737138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a947d13-1b76-471f-adec-363835960659 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-029a7a8e-e274-4264-bcbc-524131b2e17f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:20.746921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:20.748040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:20.749104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:20.749104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:20.749104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:20.749104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:20.749104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 295/594] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:20.749441 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a7a8e-e274-4264-bcbc-524131b2e17f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3a947d13-1b76-471f-adec-363835960659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:20.754784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:20.755872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:20.756946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:20.756946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:20.756946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:20.756946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:20.756946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:20.756946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:20.756946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:20.756946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:20.762790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 300/595] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:20.764282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a947d13-1b76-471f-adec-363835960659 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:20.869070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a7a8e-e274-4264-bcbc-524131b2e17f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.879640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:20.880861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:20.895722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a947d13-1b76-471f-adec-363835960659 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:20.910111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09f9c7-82f4-4ec8-aa98-38acc5442781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.910906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09f9c7-82f4-4ec8-aa98-38acc5442781 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.911200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09f9c7-82f4-4ec8-aa98-38acc5442781 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.911515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09f9c7-82f4-4ec8-aa98-38acc5442781 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.912634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09f9c7-82f4-4ec8-aa98-38acc5442781 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:20.912808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09f9c7-82f4-4ec8-aa98-38acc5442781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.914684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09f9c7-82f4-4ec8-aa98-38acc5442781 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.935569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02941b9-19e8-4a6a-89f5-a5923c9f7e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:20.936542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 296/596] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 22 11:01:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:20.936832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02941b9-19e8-4a6a-89f5-a5923c9f7e1c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:20.937398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09f9c7-82f4-4ec8-aa98-38acc5442781 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:20.937647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02941b9-19e8-4a6a-89f5-a5923c9f7e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:20.938126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02941b9-19e8-4a6a-89f5-a5923c9f7e1c None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:20.939081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02941b9-19e8-4a6a-89f5-a5923c9f7e1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:20.939622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02941b9-19e8-4a6a-89f5-a5923c9f7e1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:20.941328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02941b9-19e8-4a6a-89f5-a5923c9f7e1c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:20.956136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 301/597] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:20.957333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02941b9-19e8-4a6a-89f5-a5923c9f7e1c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:21.089432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09f9c7-82f4-4ec8-aa98-38acc5442781 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:21.091935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02941b9-19e8-4a6a-89f5-a5923c9f7e1c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:21.127004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efb76e-1961-4355-9c8d-22f3e7ccb4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:21.128023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efb76e-1961-4355-9c8d-22f3e7ccb4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.128323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efb76e-1961-4355-9c8d-22f3e7ccb4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.128590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efb76e-1961-4355-9c8d-22f3e7ccb4b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.129401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efb76e-1961-4355-9c8d-22f3e7ccb4b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:21.129720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efb76e-1961-4355-9c8d-22f3e7ccb4b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:21.131469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efb76e-1961-4355-9c8d-22f3e7ccb4b2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:21.137268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af718a4a-f10e-4bb8-b8b5-ced67cf375cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:21.138082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718a4a-f10e-4bb8-b8b5-ced67cf375cc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.138370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718a4a-f10e-4bb8-b8b5-ced67cf375cc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.138647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718a4a-f10e-4bb8-b8b5-ced67cf375cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.150391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 297/598] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:21.150391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efb76e-1961-4355-9c8d-22f3e7ccb4b2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:21.159292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af718a4a-f10e-4bb8-b8b5-ced67cf375cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:21.159292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af718a4a-f10e-4bb8-b8b5-ced67cf375cc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:21.166354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:21.168263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-af718a4a-f10e-4bb8-b8b5-ced67cf375cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:21.178757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:21.179936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:21.181031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:21.181031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:21.181031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:21.181031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:21.185058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 302/599] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:21.186327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d61563-b1fe-4fd4-9f1d-f2c412f3ca71 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.186623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d61563-b1fe-4fd4-9f1d-f2c412f3ca71 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.186908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d61563-b1fe-4fd4-9f1d-f2c412f3ca71 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.188367 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 303/600] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:21.192647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d61563-b1fe-4fd4-9f1d-f2c412f3ca71 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:21.269936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efb76e-1961-4355-9c8d-22f3e7ccb4b2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:21.297425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d61563-b1fe-4fd4-9f1d-f2c412f3ca71 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:21.308275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ff798-c244-48f9-8e7e-1bef9d2573b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:21.309075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ff798-c244-48f9-8e7e-1bef9d2573b4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.309365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ff798-c244-48f9-8e7e-1bef9d2573b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.309647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ff798-c244-48f9-8e7e-1bef9d2573b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.316948 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ff798-c244-48f9-8e7e-1bef9d2573b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:21.318437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ff798-c244-48f9-8e7e-1bef9d2573b4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:21.333872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12627685-ff1b-421e-a9cb-512cbecd7621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-464ff798-c244-48f9-8e7e-1bef9d2573b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:21.334607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:21.335504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:21.336434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:21.336434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:21.336434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:21.336434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:21.336434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:21.336434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:21.336434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:21.336434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:21.336900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12627685-ff1b-421e-a9cb-512cbecd7621 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.336900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12627685-ff1b-421e-a9cb-512cbecd7621 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.336900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12627685-ff1b-421e-a9cb-512cbecd7621 None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/c51929d0ecb047b4bdc405dc86fe7eb6/roles/a9580499eef24770867aeac1602aa002` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.337081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 298/601] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:21.337081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ff798-c244-48f9-8e7e-1bef9d2573b4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:21.354534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12627685-ff1b-421e-a9cb-512cbecd7621 None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=c51929d0ecb047b4bdc405dc86fe7eb6, role_id=a9580499eef24770867aeac1602aa002)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:21.355951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12627685-ff1b-421e-a9cb-512cbecd7621 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:21.371345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 304/602] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:01:21 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/c51929d0ecb047b4bdc405dc86fe7eb6/roles/a9580499eef24770867aeac1602aa002 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:01:21.380890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175eeb8-e724-4297-85b2-75cad0370ec0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.381241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175eeb8-e724-4297-85b2-75cad0370ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.381497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175eeb8-e724-4297-85b2-75cad0370ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.410245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e175eeb8-e724-4297-85b2-75cad0370ec0 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:21.411546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e175eeb8-e724-4297-85b2-75cad0370ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:21.439595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:21.470202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:21.485664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ff798-c244-48f9-8e7e-1bef9d2573b4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:21.507703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 305/603] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:21.508951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175eeb8-e724-4297-85b2-75cad0370ec0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:21.533463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe96c4-78ae-4c31-89fc-718b3c01d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:21.534886 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe96c4-78ae-4c31-89fc-718b3c01d046 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.535651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe96c4-78ae-4c31-89fc-718b3c01d046 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.536133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe96c4-78ae-4c31-89fc-718b3c01d046 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.537068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe96c4-78ae-4c31-89fc-718b3c01d046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:21.537752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe96c4-78ae-4c31-89fc-718b3c01d046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:21.539419 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe96c4-78ae-4c31-89fc-718b3c01d046 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:21.555194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 299/604] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:21.556363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe96c4-78ae-4c31-89fc-718b3c01d046 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:21.665284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175eeb8-e724-4297-85b2-75cad0370ec0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:21.697464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe96c4-78ae-4c31-89fc-718b3c01d046 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:21.705912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ab870-4360-4025-9471-fef52ffc3d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:21.706597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ab870-4360-4025-9471-fef52ffc3d59 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.706888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ab870-4360-4025-9471-fef52ffc3d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.707168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ab870-4360-4025-9471-fef52ffc3d59 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.708013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ab870-4360-4025-9471-fef52ffc3d59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:21.708328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ab870-4360-4025-9471-fef52ffc3d59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:21.710500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ab870-4360-4025-9471-fef52ffc3d59 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:21.726050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 306/605] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:21.727481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ab870-4360-4025-9471-fef52ffc3d59 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:21.738244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:21.746847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03b91a-332a-43c9-ab70-5005493d1590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:21.748296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03b91a-332a-43c9-ab70-5005493d1590 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.749254 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03b91a-332a-43c9-ab70-5005493d1590 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.750001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03b91a-332a-43c9-ab70-5005493d1590 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.766281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03b91a-332a-43c9-ab70-5005493d1590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6b03b91a-332a-43c9-ab70-5005493d1590 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:21.776540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:21.777504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:21.778342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:21.781896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 300/606] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:21.782074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:21.782923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03b91a-332a-43c9-ab70-5005493d1590 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:21.866035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ab870-4360-4025-9471-fef52ffc3d59 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:21.913012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b21b3-473e-4d71-9418-f07401bc4475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:21.914489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b21b3-473e-4d71-9418-f07401bc4475 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.915005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b21b3-473e-4d71-9418-f07401bc4475 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.915468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b21b3-473e-4d71-9418-f07401bc4475 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.924001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03b91a-332a-43c9-ab70-5005493d1590 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:21.925022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b21b3-473e-4d71-9418-f07401bc4475 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:21.926770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b21b3-473e-4d71-9418-f07401bc4475 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ac9b21b3-473e-4d71-9418-f07401bc4475 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:21.947622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:21.948697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:21.949581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:21.949581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:21.949581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:21.949581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:21.949581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:21.949581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:21.949581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:21.949581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:21.951949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 307/607] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:21.953018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b21b3-473e-4d71-9418-f07401bc4475 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:21.973177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb19079-6a8c-4f0e-9086-8670b22a6df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:21.973177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb19079-6a8c-4f0e-9086-8670b22a6df2 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:21.973177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb19079-6a8c-4f0e-9086-8670b22a6df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:21.973715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb19079-6a8c-4f0e-9086-8670b22a6df2 None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/0a0ce1804b514b42845ab33c7bfee309/roles/a9580499eef24770867aeac1602aa002` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:21.991174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb19079-6a8c-4f0e-9086-8670b22a6df2 None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=0a0ce1804b514b42845ab33c7bfee309, role_id=a9580499eef24770867aeac1602aa002)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:21.993024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb19079-6a8c-4f0e-9086-8670b22a6df2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:22.012040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 301/608] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:01:21 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/0a0ce1804b514b42845ab33c7bfee309/roles/a9580499eef24770867aeac1602aa002 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:01:22.013284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb19079-6a8c-4f0e-9086-8670b22a6df2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:22.074593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:22.105302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:22.109447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b21b3-473e-4d71-9418-f07401bc4475 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:22.196525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6e577-4f6d-436d-bafc-b4070356f740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:22.197130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6e577-4f6d-436d-bafc-b4070356f740 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:22.197415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6e577-4f6d-436d-bafc-b4070356f740 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:22.197748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6e577-4f6d-436d-bafc-b4070356f740 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:22.198524 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6e577-4f6d-436d-bafc-b4070356f740 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:22.198832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6e577-4f6d-436d-bafc-b4070356f740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:22.200271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6e577-4f6d-436d-bafc-b4070356f740 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:22.209541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb19079-6a8c-4f0e-9086-8670b22a6df2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:22.227165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 308/609] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:22.233381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6e577-4f6d-436d-bafc-b4070356f740 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:22.257975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110fae6-79cc-4c1e-a01e-ad24451f89d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:22.259218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110fae6-79cc-4c1e-a01e-ad24451f89d9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:22.261538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110fae6-79cc-4c1e-a01e-ad24451f89d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:22.262259 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110fae6-79cc-4c1e-a01e-ad24451f89d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:22.262647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110fae6-79cc-4c1e-a01e-ad24451f89d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:22.262959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110fae6-79cc-4c1e-a01e-ad24451f89d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:22.264793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110fae6-79cc-4c1e-a01e-ad24451f89d9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:22.281197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 302/610] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:22.288028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110fae6-79cc-4c1e-a01e-ad24451f89d9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:22.365724 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6e577-4f6d-436d-bafc-b4070356f740 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:22.399096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f9c8f-1600-4f3a-b434-0ddc2194d719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:22.399935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f9c8f-1600-4f3a-b434-0ddc2194d719 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:22.400228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f9c8f-1600-4f3a-b434-0ddc2194d719 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:22.400514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f9c8f-1600-4f3a-b434-0ddc2194d719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:22.413032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f9c8f-1600-4f3a-b434-0ddc2194d719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:22.414498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f9c8f-1600-4f3a-b434-0ddc2194d719 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-501f9c8f-1600-4f3a-b434-0ddc2194d719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:22.439913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:22.440889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:22.441785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:22.441785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:22.441785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:22.441785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:22.441785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 309/611] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:22.446796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f9c8f-1600-4f3a-b434-0ddc2194d719 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:22.471032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:22.500279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110fae6-79cc-4c1e-a01e-ad24451f89d9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:22.503231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:22.558931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ac1c-39a9-4ad9-a24b-4da4b9c57a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:22.560322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ac1c-39a9-4ad9-a24b-4da4b9c57a47 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:22.560619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ac1c-39a9-4ad9-a24b-4da4b9c57a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:22.560904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ac1c-39a9-4ad9-a24b-4da4b9c57a47 None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/5b04cb4117974f8eb9bbf3a41c48a11a/roles/a9580499eef24770867aeac1602aa002` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:22.590664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854ac1c-39a9-4ad9-a24b-4da4b9c57a47 None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=5b04cb4117974f8eb9bbf3a41c48a11a, role_id=a9580499eef24770867aeac1602aa002)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:22.594081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854ac1c-39a9-4ad9-a24b-4da4b9c57a47 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:22.626549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 303/612] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:01:22 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/5b04cb4117974f8eb9bbf3a41c48a11a/roles/a9580499eef24770867aeac1602aa002 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:01:22.663081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f9c8f-1600-4f3a-b434-0ddc2194d719 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:22.729190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00775d12-901a-42ce-affa-77a0b3c1b3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:22.730261 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00775d12-901a-42ce-affa-77a0b3c1b3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:22.730776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00775d12-901a-42ce-affa-77a0b3c1b3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:22.731280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00775d12-901a-42ce-affa-77a0b3c1b3d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:22.732261 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00775d12-901a-42ce-affa-77a0b3c1b3d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:22.732774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00775d12-901a-42ce-affa-77a0b3c1b3d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:22.734476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00775d12-901a-42ce-affa-77a0b3c1b3d0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:22.750469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 310/613] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:22.756183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ac1c-39a9-4ad9-a24b-4da4b9c57a47 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:22.819070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8428bd-6a5c-49e7-9b9b-c26214d0d8f2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:22.819422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8428bd-6a5c-49e7-9b9b-c26214d0d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:22.819673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8428bd-6a5c-49e7-9b9b-c26214d0d8f2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:22.821237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 311/614] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:01:22.827044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248938f7-849f-4ba6-b7be-cd038a3b1b7d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:22.827365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248938f7-849f-4ba6-b7be-cd038a3b1b7d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:22.827644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248938f7-849f-4ba6-b7be-cd038a3b1b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:22.860463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-248938f7-849f-4ba6-b7be-cd038a3b1b7d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:22.861729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-248938f7-849f-4ba6-b7be-cd038a3b1b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:22.900273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ac1c-39a9-4ad9-a24b-4da4b9c57a47 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:22.947483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94572fe-9474-4b03-9464-e42856c68537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:22.948661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94572fe-9474-4b03-9464-e42856c68537 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:22.948883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94572fe-9474-4b03-9464-e42856c68537 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:22.949402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94572fe-9474-4b03-9464-e42856c68537 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:22.960282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94572fe-9474-4b03-9464-e42856c68537 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:22.962141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94572fe-9474-4b03-9464-e42856c68537 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:22.970740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 312/615] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e94572fe-9474-4b03-9464-e42856c68537 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:22.978322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:22.979945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:22.981536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 304/616] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:22.986041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248938f7-849f-4ba6-b7be-cd038a3b1b7d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:22.993314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:23.000303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94572fe-9474-4b03-9464-e42856c68537 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:23.017348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:23.162751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248938f7-849f-4ba6-b7be-cd038a3b1b7d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:23.205641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1d6b6-07d4-47be-a6f2-2c9e27b03d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:23.206332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1d6b6-07d4-47be-a6f2-2c9e27b03d6e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:23.206619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1d6b6-07d4-47be-a6f2-2c9e27b03d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:23.206903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1d6b6-07d4-47be-a6f2-2c9e27b03d6e None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:23.207793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1d6b6-07d4-47be-a6f2-2c9e27b03d6e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:23.208108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1d6b6-07d4-47be-a6f2-2c9e27b03d6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:23.209638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1d6b6-07d4-47be-a6f2-2c9e27b03d6e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:23.214452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94572fe-9474-4b03-9464-e42856c68537 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:23.259762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 313/617] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 22 11:01:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:23.264431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c790dc-acbf-4bbd-9020-740c0d655e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:23.265798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c790dc-acbf-4bbd-9020-740c0d655e71 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:23.266289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c790dc-acbf-4bbd-9020-740c0d655e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:23.266695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c790dc-acbf-4bbd-9020-740c0d655e71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:23.272184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1d6b6-07d4-47be-a6f2-2c9e27b03d6e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:23.287450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c790dc-acbf-4bbd-9020-740c0d655e71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-65c790dc-acbf-4bbd-9020-740c0d655e71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:23.297637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:23.298835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:23.299970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 305/618] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:23.309088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c790dc-acbf-4bbd-9020-740c0d655e71 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:23.403745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:23.460897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:23.496658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c790dc-acbf-4bbd-9020-740c0d655e71 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:23.509228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1d6b6-07d4-47be-a6f2-2c9e27b03d6e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:23.532930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce40dd-e435-45d7-ad3a-4d1b09786200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:23.533840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce40dd-e435-45d7-ad3a-4d1b09786200 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:23.534340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce40dd-e435-45d7-ad3a-4d1b09786200 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:23.534781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce40dd-e435-45d7-ad3a-4d1b09786200 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:23.536065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce40dd-e435-45d7-ad3a-4d1b09786200 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:23.536607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce40dd-e435-45d7-ad3a-4d1b09786200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:23.538571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce40dd-e435-45d7-ad3a-4d1b09786200 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:23.549176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 306/619] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:23.554658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce40dd-e435-45d7-ad3a-4d1b09786200 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:23.631333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54470d68-ebcc-4522-bb4e-4db28c3bef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:23.632288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470d68-ebcc-4522-bb4e-4db28c3bef6a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:23.632564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470d68-ebcc-4522-bb4e-4db28c3bef6a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:23.632837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470d68-ebcc-4522-bb4e-4db28c3bef6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:23.668245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54470d68-ebcc-4522-bb4e-4db28c3bef6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:23.669743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54470d68-ebcc-4522-bb4e-4db28c3bef6a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:23.689727 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce40dd-e435-45d7-ad3a-4d1b09786200 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-54470d68-ebcc-4522-bb4e-4db28c3bef6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:23.711321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:23.712471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:23.714213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:23.714213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:23.714213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:23.714213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:23.721097 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 314/620] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:23.728018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54470d68-ebcc-4522-bb4e-4db28c3bef6a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:23.728895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2053b1-a1d6-4c29-b6c9-c1067f773219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:23.730286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2053b1-a1d6-4c29-b6c9-c1067f773219 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:23.730677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2053b1-a1d6-4c29-b6c9-c1067f773219 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:23.731074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2053b1-a1d6-4c29-b6c9-c1067f773219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:23.743625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:23.759729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2053b1-a1d6-4c29-b6c9-c1067f773219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:23.764184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2053b1-a1d6-4c29-b6c9-c1067f773219 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4d2053b1-a1d6-4c29-b6c9-c1067f773219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:23.806927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:23.808072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:23.809167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:23.809167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:23.809167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:23.809167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:23.814706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 307/621] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:23.820376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2053b1-a1d6-4c29-b6c9-c1067f773219 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:23.892533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54470d68-ebcc-4522-bb4e-4db28c3bef6a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:23.910660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:23.933394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6dc672-afb7-4598-8c62-ed2e6c77392b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:23.934065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6dc672-afb7-4598-8c62-ed2e6c77392b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:23.934353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6dc672-afb7-4598-8c62-ed2e6c77392b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:23.934633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6dc672-afb7-4598-8c62-ed2e6c77392b None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:23.935391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6dc672-afb7-4598-8c62-ed2e6c77392b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:23.935701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6dc672-afb7-4598-8c62-ed2e6c77392b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:23.937076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6dc672-afb7-4598-8c62-ed2e6c77392b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:23.950373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 315/622] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:23.951625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce080-5152-40a3-a55f-33e48297ab86 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:23.951917 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce080-5152-40a3-a55f-33e48297ab86 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:23.952451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce080-5152-40a3-a55f-33e48297ab86 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:23.957541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 316/623] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:23.958359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ce080-5152-40a3-a55f-33e48297ab86 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:23.971822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2053b1-a1d6-4c29-b6c9-c1067f773219 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:24.018938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd1d1e-afb7-48d9-b2aa-8ec348fad200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:24.020008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd1d1e-afb7-48d9-b2aa-8ec348fad200 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.021045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd1d1e-afb7-48d9-b2aa-8ec348fad200 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.021545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd1d1e-afb7-48d9-b2aa-8ec348fad200 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.022802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd1d1e-afb7-48d9-b2aa-8ec348fad200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:24.023576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd1d1e-afb7-48d9-b2aa-8ec348fad200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:24.024983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd1d1e-afb7-48d9-b2aa-8ec348fad200 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:24.041971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 308/624] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:24.043051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4fb65-1e69-4b46-b705-d9d5060cdddc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.043387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4fb65-1e69-4b46-b705-d9d5060cdddc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.043832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4fb65-1e69-4b46-b705-d9d5060cdddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.059977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0bd4fb65-1e69-4b46-b705-d9d5060cdddc None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:24.061089 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0bd4fb65-1e69-4b46-b705-d9d5060cdddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:24.087905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ce080-5152-40a3-a55f-33e48297ab86 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:24.108613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:24.136721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4378968-3a62-4a61-a254-32af97158824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:24.137873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4378968-3a62-4a61-a254-32af97158824 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.138371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4378968-3a62-4a61-a254-32af97158824 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.138877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4378968-3a62-4a61-a254-32af97158824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.143329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 309/625] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:24.144677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4fb65-1e69-4b46-b705-d9d5060cdddc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:24.160278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4378968-3a62-4a61-a254-32af97158824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:24.162108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4378968-3a62-4a61-a254-32af97158824 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a4378968-3a62-4a61-a254-32af97158824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:24.182394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:24.183413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:24.184311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:24.184311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:24.184311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:24.184311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:24.184311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:24.184311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:24.184311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:24.184311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:24.184311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 317/626] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:24.184816 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4378968-3a62-4a61-a254-32af97158824 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:24.229588 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:24.284616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4fb65-1e69-4b46-b705-d9d5060cdddc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:24.334681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4378968-3a62-4a61-a254-32af97158824 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:24.345900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411af13d-560d-4123-aac9-19bca063beb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:24.347589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411af13d-560d-4123-aac9-19bca063beb7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.348647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411af13d-560d-4123-aac9-19bca063beb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.349289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411af13d-560d-4123-aac9-19bca063beb7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.360641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411af13d-560d-4123-aac9-19bca063beb7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:24.363045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411af13d-560d-4123-aac9-19bca063beb7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:24.376548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07590ff4-a23e-48c4-ad64-6e6e19fafae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:24.377623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07590ff4-a23e-48c4-ad64-6e6e19fafae8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.378000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07590ff4-a23e-48c4-ad64-6e6e19fafae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.378441 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07590ff4-a23e-48c4-ad64-6e6e19fafae8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-411af13d-560d-4123-aac9-19bca063beb7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:24.386724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:24.387823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 22 11:01:24.388792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:24.392995 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 310/627] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:24.394107 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411af13d-560d-4123-aac9-19bca063beb7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:24.396069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07590ff4-a23e-48c4-ad64-6e6e19fafae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-07590ff4-a23e-48c4-ad64-6e6e19fafae8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:24.404982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:24.406184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:24.407202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:24.413881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 318/628] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:24.414914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07590ff4-a23e-48c4-ad64-6e6e19fafae8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:24.443002 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:24.537674 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411af13d-560d-4123-aac9-19bca063beb7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:24.538959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:24.554579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07590ff4-a23e-48c4-ad64-6e6e19fafae8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:24.591126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fe640-a289-48a4-b3ba-64dfe5d3cc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:24.600271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fe640-a289-48a4-b3ba-64dfe5d3cc2a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.600773 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fe640-a289-48a4-b3ba-64dfe5d3cc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.601258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fe640-a289-48a4-b3ba-64dfe5d3cc2a None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.602289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fe640-a289-48a4-b3ba-64dfe5d3cc2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:24.602668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34940049-4179-47ac-bd4b-ab6754778741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:24.602801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fe640-a289-48a4-b3ba-64dfe5d3cc2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:24.603684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34940049-4179-47ac-bd4b-ab6754778741 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.604144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34940049-4179-47ac-bd4b-ab6754778741 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.604568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34940049-4179-47ac-bd4b-ab6754778741 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.604743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fe640-a289-48a4-b3ba-64dfe5d3cc2a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:24.605707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34940049-4179-47ac-bd4b-ab6754778741 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:24.606168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34940049-4179-47ac-bd4b-ab6754778741 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:24.608814 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34940049-4179-47ac-bd4b-ab6754778741 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:24.619086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 311/629] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:24.620637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fe640-a289-48a4-b3ba-64dfe5d3cc2a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:24.633374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 319/630] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 22 11:01:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:24.634421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34940049-4179-47ac-bd4b-ab6754778741 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:24.743724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:24.749168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34940049-4179-47ac-bd4b-ab6754778741 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:24.753729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fe640-a289-48a4-b3ba-64dfe5d3cc2a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:24.793397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f59cd-d2dd-4da7-b529-2a866563b200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:24.794303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f59cd-d2dd-4da7-b529-2a866563b200 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.794594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f59cd-d2dd-4da7-b529-2a866563b200 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.794875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f59cd-d2dd-4da7-b529-2a866563b200 None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/e7935e85cc3f4ed1ad5b01ba892e6e82/roles/a9580499eef24770867aeac1602aa002` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.798405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2bba6-124e-49f8-87fa-42b103803511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:24.799855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2bba6-124e-49f8-87fa-42b103803511 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.800150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2bba6-124e-49f8-87fa-42b103803511 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.800436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2bba6-124e-49f8-87fa-42b103803511 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.801224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2bba6-124e-49f8-87fa-42b103803511 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:24.801541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2bba6-124e-49f8-87fa-42b103803511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:24.802963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2bba6-124e-49f8-87fa-42b103803511 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:24.812909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 312/631] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:24.814234 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856329a2-3370-4717-9480-43e648e94ef3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:24.814531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856329a2-3370-4717-9480-43e648e94ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:24.814817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856329a2-3370-4717-9480-43e648e94ef3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:24.819442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 313/632] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:24.819442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856329a2-3370-4717-9480-43e648e94ef3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:24.821088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f59cd-d2dd-4da7-b529-2a866563b200 None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=e7935e85cc3f4ed1ad5b01ba892e6e82, role_id=a9580499eef24770867aeac1602aa002)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:24.822555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f59cd-d2dd-4da7-b529-2a866563b200 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:24.837733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 320/633] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:01:24 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/e7935e85cc3f4ed1ad5b01ba892e6e82/roles/a9580499eef24770867aeac1602aa002 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:01:24.838826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f59cd-d2dd-4da7-b529-2a866563b200 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:24.848283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:24.957982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856329a2-3370-4717-9480-43e648e94ef3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:24.961675 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f59cd-d2dd-4da7-b529-2a866563b200 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:25.003923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfaba0d-f6f2-40e6-b691-00fbd0514931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:25.004138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433c55b-a2ff-48f7-8f48-b0d35dc793f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:25.004687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433c55b-a2ff-48f7-8f48-b0d35dc793f6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.005014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaba0d-f6f2-40e6-b691-00fbd0514931 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.005139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433c55b-a2ff-48f7-8f48-b0d35dc793f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.005334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433c55b-a2ff-48f7-8f48-b0d35dc793f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.005428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaba0d-f6f2-40e6-b691-00fbd0514931 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.005770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfaba0d-f6f2-40e6-b691-00fbd0514931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.018559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfaba0d-f6f2-40e6-b691-00fbd0514931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:25.020184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfaba0d-f6f2-40e6-b691-00fbd0514931 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:25.036870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ecfaba0d-f6f2-40e6-b691-00fbd0514931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:25.040978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:25.042488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:25.043440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:25.043440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:25.043440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:25.043440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:25.043440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 321/634] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:25.044373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909d117-3873-4caa-a060-8654c85f981d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.044612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909d117-3873-4caa-a060-8654c85f981d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.044894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909d117-3873-4caa-a060-8654c85f981d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.066763 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7909d117-3873-4caa-a060-8654c85f981d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:25.068016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7909d117-3873-4caa-a060-8654c85f981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:25.099399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433c55b-a2ff-48f7-8f48-b0d35dc793f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:25.099767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433c55b-a2ff-48f7-8f48-b0d35dc793f6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:25.129007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:25.149212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 322/635] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:25.150494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909d117-3873-4caa-a060-8654c85f981d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:25.226969 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 314/636] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:25.228568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada2022-6b74-4add-84e5-ef55c32397c4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.229546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada2022-6b74-4add-84e5-ef55c32397c4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.229828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada2022-6b74-4add-84e5-ef55c32397c4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.231712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 315/637] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:25.233368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada2022-6b74-4add-84e5-ef55c32397c4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:25.264643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909d117-3873-4caa-a060-8654c85f981d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:25.301525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f7243-2ee7-47ea-9437-5b0dfab91935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:25.302204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f7243-2ee7-47ea-9437-5b0dfab91935 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.302516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f7243-2ee7-47ea-9437-5b0dfab91935 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.302775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f7243-2ee7-47ea-9437-5b0dfab91935 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.303532 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f7243-2ee7-47ea-9437-5b0dfab91935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:25.303836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f7243-2ee7-47ea-9437-5b0dfab91935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:25.305207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f7243-2ee7-47ea-9437-5b0dfab91935 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:25.305617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:25.318125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 323/638] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:25.318769 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f7243-2ee7-47ea-9437-5b0dfab91935 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:25.361345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada2022-6b74-4add-84e5-ef55c32397c4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:25.374237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:25.407170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb9b91-dde8-4aed-953d-58b5db4aa69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:25.408010 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9b91-dde8-4aed-953d-58b5db4aa69f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.408297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9b91-dde8-4aed-953d-58b5db4aa69f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.408579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9b91-dde8-4aed-953d-58b5db4aa69f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.420781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb9b91-dde8-4aed-953d-58b5db4aa69f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f4bb9b91-dde8-4aed-953d-58b5db4aa69f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:25.428569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:25.429716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:25.430589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 316/639] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:25.431398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c11229-125a-4852-9b12-466f8bf371d9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.431624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c11229-125a-4852-9b12-466f8bf371d9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.431972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c11229-125a-4852-9b12-466f8bf371d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.445921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f7243-2ee7-47ea-9437-5b0dfab91935 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:25.448842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b3c11229-125a-4852-9b12-466f8bf371d9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:25.450254 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b3c11229-125a-4852-9b12-466f8bf371d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:25.491452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1c4cf-c451-4ee3-a36f-951477052fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:25.492259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1c4cf-c451-4ee3-a36f-951477052fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.492551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1c4cf-c451-4ee3-a36f-951477052fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.492831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1c4cf-c451-4ee3-a36f-951477052fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.504823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1c4cf-c451-4ee3-a36f-951477052fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:25.506505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1c4cf-c451-4ee3-a36f-951477052fb1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:25.522981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 317/640] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:25.524179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c11229-125a-4852-9b12-466f8bf371d9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-42f1c4cf-c451-4ee3-a36f-951477052fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:25.525202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:25.526605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:25.527550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:25.527550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:25.527550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:25.527550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:25.531602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 324/641] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:25.532370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1c4cf-c451-4ee3-a36f-951477052fb1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:25.582552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:25.642505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c11229-125a-4852-9b12-466f8bf371d9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:25.652011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:25.659000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1c4cf-c451-4ee3-a36f-951477052fb1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:25.682126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373420e-382b-4804-9153-89400c4d5374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:25.682946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373420e-382b-4804-9153-89400c4d5374 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.683221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373420e-382b-4804-9153-89400c4d5374 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.683528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373420e-382b-4804-9153-89400c4d5374 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.691532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373420e-382b-4804-9153-89400c4d5374 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:25.692991 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373420e-382b-4804-9153-89400c4d5374 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9373420e-382b-4804-9153-89400c4d5374 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:25.708386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:25.718167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:25.720045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:25.721019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481709c-d8d5-4973-95e3-504bac012603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:25.721019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481709c-d8d5-4973-95e3-504bac012603 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.721019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481709c-d8d5-4973-95e3-504bac012603 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.721379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 318/642] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 22 11:01:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:25.721495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481709c-d8d5-4973-95e3-504bac012603 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.721615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373420e-382b-4804-9153-89400c4d5374 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:25.721745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481709c-d8d5-4973-95e3-504bac012603 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:25.721745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481709c-d8d5-4973-95e3-504bac012603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:25.721745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481709c-d8d5-4973-95e3-504bac012603 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:25.735944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 325/643] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:25.735944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481709c-d8d5-4973-95e3-504bac012603 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:25.839501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373420e-382b-4804-9153-89400c4d5374 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:25.849956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:25.860498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481709c-d8d5-4973-95e3-504bac012603 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:25.880797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeaa79-1177-4151-9d4e-e49861cc76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:25.881478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeaa79-1177-4151-9d4e-e49861cc76b9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.881835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeaa79-1177-4151-9d4e-e49861cc76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.882085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeaa79-1177-4151-9d4e-e49861cc76b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.882817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeaa79-1177-4151-9d4e-e49861cc76b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:25.883071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeaa79-1177-4151-9d4e-e49861cc76b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:25.884503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeaa79-1177-4151-9d4e-e49861cc76b9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:25.905222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 319/644] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:25.905222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeaa79-1177-4151-9d4e-e49861cc76b9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:25.912203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d5083-5017-4866-810f-82cf47eaba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:25.913607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d5083-5017-4866-810f-82cf47eaba19 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:25.914132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d5083-5017-4866-810f-82cf47eaba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:25.914664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d5083-5017-4866-810f-82cf47eaba19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:25.935984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d5083-5017-4866-810f-82cf47eaba19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8e3d5083-5017-4866-810f-82cf47eaba19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:25.949515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:25.950505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:25.951535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 326/645] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:25.953160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d5083-5017-4866-810f-82cf47eaba19 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:25.983575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:26.040192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeaa79-1177-4151-9d4e-e49861cc76b9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:26.075003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d5083-5017-4866-810f-82cf47eaba19 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:26.080362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574a30f-035a-4df4-86c7-a955a6f85811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:26.081081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574a30f-035a-4df4-86c7-a955a6f85811 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:26.081378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574a30f-035a-4df4-86c7-a955a6f85811 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:26.081662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574a30f-035a-4df4-86c7-a955a6f85811 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:26.082552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574a30f-035a-4df4-86c7-a955a6f85811 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:26.082870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574a30f-035a-4df4-86c7-a955a6f85811 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:26.084764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574a30f-035a-4df4-86c7-a955a6f85811 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:26.108116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 320/646] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 22 11:01:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:26.108962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574a30f-035a-4df4-86c7-a955a6f85811 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:26.126295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5b922-a118-4eb8-9d8f-7e135a73f4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:26.127104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5b922-a118-4eb8-9d8f-7e135a73f4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:26.127427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5b922-a118-4eb8-9d8f-7e135a73f4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:26.127711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5b922-a118-4eb8-9d8f-7e135a73f4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:26.140829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5b922-a118-4eb8-9d8f-7e135a73f4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:26.142369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5b922-a118-4eb8-9d8f-7e135a73f4d5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:26.147065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4ba5b922-a118-4eb8-9d8f-7e135a73f4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:26.161235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:26.162151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:26.168891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:26.168891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:26.168891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:26.168891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:26.168891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 327/647] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:26.168891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5b922-a118-4eb8-9d8f-7e135a73f4d5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:26.244445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:26.249930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574a30f-035a-4df4-86c7-a955a6f85811 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:26.278616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5b922-a118-4eb8-9d8f-7e135a73f4d5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:26.290414 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52969f56-3f30-4297-8c3d-5fd8544d63c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:26.292086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52969f56-3f30-4297-8c3d-5fd8544d63c4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:26.292372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52969f56-3f30-4297-8c3d-5fd8544d63c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:26.292648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52969f56-3f30-4297-8c3d-5fd8544d63c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:26.301484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52969f56-3f30-4297-8c3d-5fd8544d63c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:26.303397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52969f56-3f30-4297-8c3d-5fd8544d63c4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:26.314468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e759ec-a715-423c-a0bb-1c7e664f5489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:26.315129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e759ec-a715-423c-a0bb-1c7e664f5489 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:26.315452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e759ec-a715-423c-a0bb-1c7e664f5489 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:26.315735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e759ec-a715-423c-a0bb-1c7e664f5489 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:26.316482 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e759ec-a715-423c-a0bb-1c7e664f5489 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:26.316788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e759ec-a715-423c-a0bb-1c7e664f5489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:26.318165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e759ec-a715-423c-a0bb-1c7e664f5489 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-52969f56-3f30-4297-8c3d-5fd8544d63c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:26.319786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:26.320857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 321/648] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:26.328075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52969f56-3f30-4297-8c3d-5fd8544d63c4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:26.331612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 328/649] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:26.332508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e759ec-a715-423c-a0bb-1c7e664f5489 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:26.425246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:26.442625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52969f56-3f30-4297-8c3d-5fd8544d63c4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:26.476139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e759ec-a715-423c-a0bb-1c7e664f5489 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:26.479503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73973652-316a-4667-871d-7f74c551db89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:26.480168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73973652-316a-4667-871d-7f74c551db89 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:26.480446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73973652-316a-4667-871d-7f74c551db89 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:26.480725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73973652-316a-4667-871d-7f74c551db89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:26.515540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:26.522110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c05831-5843-4ae5-a198-5d934141c7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:26.522980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c05831-5843-4ae5-a198-5d934141c7bc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:26.523507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c05831-5843-4ae5-a198-5d934141c7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:26.523791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c05831-5843-4ae5-a198-5d934141c7bc None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:26.524526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c05831-5843-4ae5-a198-5d934141c7bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:26.524842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c05831-5843-4ae5-a198-5d934141c7bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:26.526281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c05831-5843-4ae5-a198-5d934141c7bc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:26.544393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 329/650] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:26.544393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c05831-5843-4ae5-a198-5d934141c7bc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:26.566734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73973652-316a-4667-871d-7f74c551db89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:26.568179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73973652-316a-4667-871d-7f74c551db89 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:26.651019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c05831-5843-4ae5-a198-5d934141c7bc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:26.660654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:26.687533 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 322/651] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:26.688311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73973652-316a-4667-871d-7f74c551db89 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:26.706443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691d684-dfb8-4c83-ab62-0a33403e713c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:26.707129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691d684-dfb8-4c83-ab62-0a33403e713c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:26.707464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691d684-dfb8-4c83-ab62-0a33403e713c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:26.707746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691d684-dfb8-4c83-ab62-0a33403e713c None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:26.708547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691d684-dfb8-4c83-ab62-0a33403e713c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:26.708863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691d684-dfb8-4c83-ab62-0a33403e713c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:26.710319 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691d684-dfb8-4c83-ab62-0a33403e713c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:26.723482 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 330/652] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:26.724156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691d684-dfb8-4c83-ab62-0a33403e713c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:26.783111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:26.807343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73973652-316a-4667-871d-7f74c551db89 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:26.834465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691d684-dfb8-4c83-ab62-0a33403e713c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:26.844519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb58af7-6c02-44b7-91a9-3e4e6be6f078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:26.845313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb58af7-6c02-44b7-91a9-3e4e6be6f078 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:26.845595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb58af7-6c02-44b7-91a9-3e4e6be6f078 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:26.845870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb58af7-6c02-44b7-91a9-3e4e6be6f078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:26.862019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb58af7-6c02-44b7-91a9-3e4e6be6f078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:26.863522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb58af7-6c02-44b7-91a9-3e4e6be6f078 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:26.878252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792d4c-9a2c-45ee-8b9c-fb2a03171ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:26.879064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792d4c-9a2c-45ee-8b9c-fb2a03171ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:26.879394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792d4c-9a2c-45ee-8b9c-fb2a03171ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:26.879675 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792d4c-9a2c-45ee-8b9c-fb2a03171ed7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-aeb58af7-6c02-44b7-91a9-3e4e6be6f078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:26.884187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:26.885066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:26.886041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:26.886041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:26.886041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:26.886041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:26.886041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 323/653] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:26.886877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb58af7-6c02-44b7-91a9-3e4e6be6f078 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:26.892560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb792d4c-9a2c-45ee-8b9c-fb2a03171ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:26.894030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb792d4c-9a2c-45ee-8b9c-fb2a03171ed7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fb792d4c-9a2c-45ee-8b9c-fb2a03171ed7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:26.909522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:26.910525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:26.911488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 331/654] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:26.912548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792d4c-9a2c-45ee-8b9c-fb2a03171ed7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:26.938894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:27.012166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb58af7-6c02-44b7-91a9-3e4e6be6f078 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:27.031114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792d4c-9a2c-45ee-8b9c-fb2a03171ed7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:27.046259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:27.053183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ca8fe-29e8-46c2-8f54-2e7d00e79b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:27.054213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca8fe-29e8-46c2-8f54-2e7d00e79b1e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.054665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca8fe-29e8-46c2-8f54-2e7d00e79b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.055112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca8fe-29e8-46c2-8f54-2e7d00e79b1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.066895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ca8fe-29e8-46c2-8f54-2e7d00e79b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:27.068522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ca8fe-29e8-46c2-8f54-2e7d00e79b1e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:27.070014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb34cb-1ff0-487b-865e-166c63ba267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:27.070680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb34cb-1ff0-487b-865e-166c63ba267a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.070968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb34cb-1ff0-487b-865e-166c63ba267a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.071249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb34cb-1ff0-487b-865e-166c63ba267a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.072119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb34cb-1ff0-487b-865e-166c63ba267a None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=92fc25f9ad7c49c48a909d2d92ea0a66) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:27.072960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb34cb-1ff0-487b-865e-166c63ba267a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:27.074393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb34cb-1ff0-487b-865e-166c63ba267a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:27.080406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 332/655] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:26 2022] GET /identity/v3/role_assignments?user.id=92fc25f9ad7c49c48a909d2d92ea0a66&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:27.080999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb34cb-1ff0-487b-865e-166c63ba267a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-788ca8fe-29e8-46c2-8f54-2e7d00e79b1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:27.086860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:27.088571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:27.089708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:27.089708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:27.089708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:27.089708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:27.089708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 324/656] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:27.089708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ca8fe-29e8-46c2-8f54-2e7d00e79b1e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:27.192571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:27.205657 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ca8fe-29e8-46c2-8f54-2e7d00e79b1e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:27.210808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb34cb-1ff0-487b-865e-166c63ba267a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:27.251811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1845f-010b-4cfb-b2f9-ff9b76e62523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:27.252496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1845f-010b-4cfb-b2f9-ff9b76e62523 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.252785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1845f-010b-4cfb-b2f9-ff9b76e62523 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.253007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cbf65-bcf2-4156-bbe9-8a2bf379d52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:27.253124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1845f-010b-4cfb-b2f9-ff9b76e62523 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.254122 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbf65-bcf2-4156-bbe9-8a2bf379d52f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.254229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1845f-010b-4cfb-b2f9-ff9b76e62523 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:27.254305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbf65-bcf2-4156-bbe9-8a2bf379d52f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.254387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1845f-010b-4cfb-b2f9-ff9b76e62523 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:27.254459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbf65-bcf2-4156-bbe9-8a2bf379d52f None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.254938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cbf65-bcf2-4156-bbe9-8a2bf379d52f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:27.255245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cbf65-bcf2-4156-bbe9-8a2bf379d52f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:27.255685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1845f-010b-4cfb-b2f9-ff9b76e62523 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:27.257450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cbf65-bcf2-4156-bbe9-8a2bf379d52f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:27.271558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 333/657] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:27.272960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef90c9-1565-4250-b73a-21a824f7cac8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.273266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef90c9-1565-4250-b73a-21a824f7cac8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.273548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef90c9-1565-4250-b73a-21a824f7cac8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.274964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 334/658] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:27.276353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef90c9-1565-4250-b73a-21a824f7cac8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:27.283913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 325/659] 10.222.0.20 () {56 vars in 1139 bytes} [Fri Jul 22 11:01:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:27.283913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1845f-010b-4cfb-b2f9-ff9b76e62523 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:27.326561 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:27.401583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef90c9-1565-4250-b73a-21a824f7cac8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:27.405876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1845f-010b-4cfb-b2f9-ff9b76e62523 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:27.438180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f57e2-cf48-483c-856b-0ffa11904f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:27.438852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f57e2-cf48-483c-856b-0ffa11904f38 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.439143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f57e2-cf48-483c-856b-0ffa11904f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.439619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f57e2-cf48-483c-856b-0ffa11904f38 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.440250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f57e2-cf48-483c-856b-0ffa11904f38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:27.440483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f57e2-cf48-483c-856b-0ffa11904f38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:27.441901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f57e2-cf48-483c-856b-0ffa11904f38 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:27.448182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1072-f0db-4a64-b3fe-a6de9067fd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:27.448919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1072-f0db-4a64-b3fe-a6de9067fd6c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.449206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1072-f0db-4a64-b3fe-a6de9067fd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.449478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1072-f0db-4a64-b3fe-a6de9067fd6c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.455061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 335/660] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:27.456790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1072-f0db-4a64-b3fe-a6de9067fd6c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:27.457063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619cf33-b120-4c49-84c2-74e2d34b9ecc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.457549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619cf33-b120-4c49-84c2-74e2d34b9ecc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.457999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619cf33-b120-4c49-84c2-74e2d34b9ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.458291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1072-f0db-4a64-b3fe-a6de9067fd6c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4fca1072-f0db-4a64-b3fe-a6de9067fd6c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:27.473923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:27.474937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 326/661] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:27.476408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1072-f0db-4a64-b3fe-a6de9067fd6c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:27.480846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4619cf33-b120-4c49-84c2-74e2d34b9ecc None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:27.482500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4619cf33-b120-4c49-84c2-74e2d34b9ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:27.490607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:27.572966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 336/662] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:27.580582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619cf33-b120-4c49-84c2-74e2d34b9ecc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:27.615858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:27.617848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1072-f0db-4a64-b3fe-a6de9067fd6c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:27.672773 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853bddd-6055-480f-ac33-22f2679b476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:27.673441 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853bddd-6055-480f-ac33-22f2679b476f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.673735 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853bddd-6055-480f-ac33-22f2679b476f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.674019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853bddd-6055-480f-ac33-22f2679b476f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.710908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619cf33-b120-4c49-84c2-74e2d34b9ecc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:27.746926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11cc5-e002-4d1c-bd58-9617c7f6d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:27.747764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11cc5-e002-4d1c-bd58-9617c7f6d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.748058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11cc5-e002-4d1c-bd58-9617c7f6d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.748401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11cc5-e002-4d1c-bd58-9617c7f6d8b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.755357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11cc5-e002-4d1c-bd58-9617c7f6d8b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:27.755505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853bddd-6055-480f-ac33-22f2679b476f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:27.756821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11cc5-e002-4d1c-bd58-9617c7f6d8b8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:27.757011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853bddd-6055-480f-ac33-22f2679b476f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:27.768115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b3a11cc5-e002-4d1c-bd58-9617c7f6d8b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:27.776352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:27.777244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:27.778146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:27.778146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:27.778146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:27.778146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:27.778146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:27.778146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:27.778146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:27.778146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:27.778146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 337/663] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:27.778642 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11cc5-e002-4d1c-bd58-9617c7f6d8b8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:27.874930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 327/664] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:27.876468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853bddd-6055-480f-ac33-22f2679b476f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:27.884962 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:27.905736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11cc5-e002-4d1c-bd58-9617c7f6d8b8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:27.944217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95273cf-6ec4-4604-8ce1-8418811b6fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:27.944963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95273cf-6ec4-4604-8ce1-8418811b6fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:27.945191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95273cf-6ec4-4604-8ce1-8418811b6fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:27.945478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95273cf-6ec4-4604-8ce1-8418811b6fc0 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:27.946326 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95273cf-6ec4-4604-8ce1-8418811b6fc0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:27.946634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95273cf-6ec4-4604-8ce1-8418811b6fc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:27.948694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95273cf-6ec4-4604-8ce1-8418811b6fc0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:27.974574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 338/665] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Jul 22 11:01:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:27.974574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95273cf-6ec4-4604-8ce1-8418811b6fc0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:27.995987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853bddd-6055-480f-ac33-22f2679b476f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.005564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:28.030800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7278-054e-452b-8ff2-c188b250881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.031634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7278-054e-452b-8ff2-c188b250881c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.031926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7278-054e-452b-8ff2-c188b250881c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.032212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7278-054e-452b-8ff2-c188b250881c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.043503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b7278-054e-452b-8ff2-c188b250881c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-597b7278-054e-452b-8ff2-c188b250881c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:28.051348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:28.052289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:28.053131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 328/666] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:28.053868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7278-054e-452b-8ff2-c188b250881c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:28.082551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95273cf-6ec4-4604-8ce1-8418811b6fc0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.124727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65f4f7-9a44-47f8-b12a-e9bcb4b19d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.125386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65f4f7-9a44-47f8-b12a-e9bcb4b19d18 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.125675 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65f4f7-9a44-47f8-b12a-e9bcb4b19d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.125955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65f4f7-9a44-47f8-b12a-e9bcb4b19d18 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.126738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65f4f7-9a44-47f8-b12a-e9bcb4b19d18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:28.127106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65f4f7-9a44-47f8-b12a-e9bcb4b19d18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:28.128639 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65f4f7-9a44-47f8-b12a-e9bcb4b19d18 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:28.142564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 339/667] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:28.143689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65f4f7-9a44-47f8-b12a-e9bcb4b19d18 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:28.150541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:28.171593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7278-054e-452b-8ff2-c188b250881c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.206967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc978e-9a2f-4b04-9cf0-31c6828cba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.208149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc978e-9a2f-4b04-9cf0-31c6828cba2b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.208631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc978e-9a2f-4b04-9cf0-31c6828cba2b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.208945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc978e-9a2f-4b04-9cf0-31c6828cba2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.220292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc978e-9a2f-4b04-9cf0-31c6828cba2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:28.221747 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc978e-9a2f-4b04-9cf0-31c6828cba2b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6dcc978e-9a2f-4b04-9cf0-31c6828cba2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:28.239875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:28.240767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:28.241689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:28.241689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:28.241689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:28.241689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:28.241689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 329/668] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:28.241974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc978e-9a2f-4b04-9cf0-31c6828cba2b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:28.260017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65f4f7-9a44-47f8-b12a-e9bcb4b19d18 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.266839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:28.296424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9f067-2606-4e08-a2c6-720fcbc67b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.297043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9f067-2606-4e08-a2c6-720fcbc67b2b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.297331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9f067-2606-4e08-a2c6-720fcbc67b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.297614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9f067-2606-4e08-a2c6-720fcbc67b2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.349983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc978e-9a2f-4b04-9cf0-31c6828cba2b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.371536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9f067-2606-4e08-a2c6-720fcbc67b2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:28.373003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9f067-2606-4e08-a2c6-720fcbc67b2b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:28.375745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:28.386609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea73ce-220c-4b19-969f-b33f511f7d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.387302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea73ce-220c-4b19-969f-b33f511f7d12 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.387595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea73ce-220c-4b19-969f-b33f511f7d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.387874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea73ce-220c-4b19-969f-b33f511f7d12 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.388625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea73ce-220c-4b19-969f-b33f511f7d12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:28.388933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea73ce-220c-4b19-969f-b33f511f7d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:28.390301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea73ce-220c-4b19-969f-b33f511f7d12 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:28.403156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 330/669] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:28.403156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea73ce-220c-4b19-969f-b33f511f7d12 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:28.482746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 340/670] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:28.483714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9f067-2606-4e08-a2c6-720fcbc67b2b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:28.493512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:28.515721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea73ce-220c-4b19-969f-b33f511f7d12 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.552325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9500b-c6d6-4e48-9408-a30c9e1b5488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.553296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9500b-c6d6-4e48-9408-a30c9e1b5488 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.553734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9500b-c6d6-4e48-9408-a30c9e1b5488 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.554161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9500b-c6d6-4e48-9408-a30c9e1b5488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.555163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9500b-c6d6-4e48-9408-a30c9e1b5488 None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=c51929d0ecb047b4bdc405dc86fe7eb6) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:28.555731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9500b-c6d6-4e48-9408-a30c9e1b5488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:28.557480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9500b-c6d6-4e48-9408-a30c9e1b5488 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:28.571482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 331/671] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/role_assignments?user.id=c51929d0ecb047b4bdc405dc86fe7eb6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:28.573021 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9500b-c6d6-4e48-9408-a30c9e1b5488 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:28.589983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9f067-2606-4e08-a2c6-720fcbc67b2b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.598992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:28.636108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4646b48-2a11-4b31-a8ec-e43c6088a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.636896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4646b48-2a11-4b31-a8ec-e43c6088a917 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.637189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4646b48-2a11-4b31-a8ec-e43c6088a917 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.643282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4646b48-2a11-4b31-a8ec-e43c6088a917 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.643282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4646b48-2a11-4b31-a8ec-e43c6088a917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:28.643282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4646b48-2a11-4b31-a8ec-e43c6088a917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:28.643282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4646b48-2a11-4b31-a8ec-e43c6088a917 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:28.659087 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 341/672] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:28.660025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4646b48-2a11-4b31-a8ec-e43c6088a917 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:28.708566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9500b-c6d6-4e48-9408-a30c9e1b5488 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.754420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac03ab-0d17-4b55-9ea5-7c08b155e851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.755496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac03ab-0d17-4b55-9ea5-7c08b155e851 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.755788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac03ab-0d17-4b55-9ea5-7c08b155e851 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.756071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac03ab-0d17-4b55-9ea5-7c08b155e851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.768729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac03ab-0d17-4b55-9ea5-7c08b155e851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:28.770205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac03ab-0d17-4b55-9ea5-7c08b155e851 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:28.778252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:28.782379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4646b48-2a11-4b31-a8ec-e43c6088a917 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-08ac03ab-0d17-4b55-9ea5-7c08b155e851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:28.790251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:28.791235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:28.792179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:28.792179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:28.792179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:28.792179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:28.792179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 332/673] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:28.793325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac03ab-0d17-4b55-9ea5-7c08b155e851 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:28.827857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb996d24-e7f2-4572-960f-4793790239c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.828691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb996d24-e7f2-4572-960f-4793790239c3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.828949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb996d24-e7f2-4572-960f-4793790239c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.829306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb996d24-e7f2-4572-960f-4793790239c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.843425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb996d24-e7f2-4572-960f-4793790239c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:28.845385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb996d24-e7f2-4572-960f-4793790239c3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fb996d24-e7f2-4572-960f-4793790239c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:28.864947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:28.865888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:28.866853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:28.866853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:28.866853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:28.866853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:28.866853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 342/674] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:28.867137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb996d24-e7f2-4572-960f-4793790239c3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:28.888932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:28.926126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac03ab-0d17-4b55-9ea5-7c08b155e851 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:28.966032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77841cc-b60e-4645-8c95-6dc9a00c56e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:28.966432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77841cc-b60e-4645-8c95-6dc9a00c56e1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:28.966724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77841cc-b60e-4645-8c95-6dc9a00c56e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:28.967004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77841cc-b60e-4645-8c95-6dc9a00c56e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:28.974714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77841cc-b60e-4645-8c95-6dc9a00c56e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:28.976215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77841cc-b60e-4645-8c95-6dc9a00c56e1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e77841cc-b60e-4645-8c95-6dc9a00c56e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:28.991428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:28.992385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:28.993285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:28.993285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:28.993285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:28.993285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:28.993285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:28.993285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:28.993285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:28.993285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:28.993285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 333/675] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:28.994846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77841cc-b60e-4645-8c95-6dc9a00c56e1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:29.009453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb996d24-e7f2-4572-960f-4793790239c3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:29.061871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2e22b-8e7e-4d3b-a807-e53b4f04a547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:29.063743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2e22b-8e7e-4d3b-a807-e53b4f04a547 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.065225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2e22b-8e7e-4d3b-a807-e53b4f04a547 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.066320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2e22b-8e7e-4d3b-a807-e53b4f04a547 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.068978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2e22b-8e7e-4d3b-a807-e53b4f04a547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:29.069556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2e22b-8e7e-4d3b-a807-e53b4f04a547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:29.073201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2e22b-8e7e-4d3b-a807-e53b4f04a547 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:29.079556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:29.093035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 343/676] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:29.094408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2e22b-8e7e-4d3b-a807-e53b4f04a547 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:29.153236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77841cc-b60e-4645-8c95-6dc9a00c56e1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:29.202634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:29.209169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701908e-1877-4cbb-9279-b05a42222b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:29.210212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701908e-1877-4cbb-9279-b05a42222b62 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.210720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701908e-1877-4cbb-9279-b05a42222b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.211394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701908e-1877-4cbb-9279-b05a42222b62 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.214098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701908e-1877-4cbb-9279-b05a42222b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:29.214956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701908e-1877-4cbb-9279-b05a42222b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:29.217239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701908e-1877-4cbb-9279-b05a42222b62 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:29.233167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2e22b-8e7e-4d3b-a807-e53b4f04a547 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:29.238963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 334/677] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:29.240663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701908e-1877-4cbb-9279-b05a42222b62 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:29.285903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db485b-df53-4cac-b734-8a03c41f7d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:29.286844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db485b-df53-4cac-b734-8a03c41f7d31 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.287239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db485b-df53-4cac-b734-8a03c41f7d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.287682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db485b-df53-4cac-b734-8a03c41f7d31 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.288878 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db485b-df53-4cac-b734-8a03c41f7d31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:29.289405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db485b-df53-4cac-b734-8a03c41f7d31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:29.292511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db485b-df53-4cac-b734-8a03c41f7d31 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:29.313559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 344/678] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:29.313559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db485b-df53-4cac-b734-8a03c41f7d31 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:29.390550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701908e-1877-4cbb-9279-b05a42222b62 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:29.417778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:29.441040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2168c-69d6-42c9-b574-ba3f3348118e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:29.442010 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2168c-69d6-42c9-b574-ba3f3348118e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.442293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2168c-69d6-42c9-b574-ba3f3348118e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.442617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2168c-69d6-42c9-b574-ba3f3348118e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.451621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db485b-df53-4cac-b734-8a03c41f7d31 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:29.455873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2168c-69d6-42c9-b574-ba3f3348118e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:29.458280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2168c-69d6-42c9-b574-ba3f3348118e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b3b2168c-69d6-42c9-b574-ba3f3348118e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:29.477964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:29.479564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:29.480519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 335/679] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:29.481115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c305845-ddc3-47c9-971a-b1e0c03b4ae1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.481857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c305845-ddc3-47c9-971a-b1e0c03b4ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.481857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c305845-ddc3-47c9-971a-b1e0c03b4ae1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.483120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 336/680] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:29.486495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c305845-ddc3-47c9-971a-b1e0c03b4ae1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:29.512298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3762c9f-0ffc-43f0-b970-9b0e62a1f092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:29.513361 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3762c9f-0ffc-43f0-b970-9b0e62a1f092 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.513843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3762c9f-0ffc-43f0-b970-9b0e62a1f092 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.514134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3762c9f-0ffc-43f0-b970-9b0e62a1f092 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.518547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:29.524016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3762c9f-0ffc-43f0-b970-9b0e62a1f092 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:29.525693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3762c9f-0ffc-43f0-b970-9b0e62a1f092 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f3762c9f-0ffc-43f0-b970-9b0e62a1f092 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:29.542451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:29.543562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 345/681] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:29.545196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3762c9f-0ffc-43f0-b970-9b0e62a1f092 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:29.614554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c305845-ddc3-47c9-971a-b1e0c03b4ae1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:29.656368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828bd8f-61ac-49f6-8d5f-93c62434058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:29.657426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828bd8f-61ac-49f6-8d5f-93c62434058a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.657996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828bd8f-61ac-49f6-8d5f-93c62434058a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.658287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828bd8f-61ac-49f6-8d5f-93c62434058a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.659316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828bd8f-61ac-49f6-8d5f-93c62434058a None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=0a0ce1804b514b42845ab33c7bfee309) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:29.659643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828bd8f-61ac-49f6-8d5f-93c62434058a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:29.661709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828bd8f-61ac-49f6-8d5f-93c62434058a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:29.662641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3762c9f-0ffc-43f0-b970-9b0e62a1f092 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:29.679293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 337/682] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:29 2022] GET /identity/v3/role_assignments?user.id=0a0ce1804b514b42845ab33c7bfee309&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:29.679293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed334bc9-ce31-47d2-b02a-1c57d42bc19f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.679293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed334bc9-ce31-47d2-b02a-1c57d42bc19f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.679293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed334bc9-ce31-47d2-b02a-1c57d42bc19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.701651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ed334bc9-ce31-47d2-b02a-1c57d42bc19f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:29.702931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ed334bc9-ce31-47d2-b02a-1c57d42bc19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:29.704341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ad63b-478c-4c02-8d65-72e36013194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:29.705519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ad63b-478c-4c02-8d65-72e36013194b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.706245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ad63b-478c-4c02-8d65-72e36013194b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.706651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ad63b-478c-4c02-8d65-72e36013194b None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.708230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ad63b-478c-4c02-8d65-72e36013194b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:29.708558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ad63b-478c-4c02-8d65-72e36013194b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:29.710079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ad63b-478c-4c02-8d65-72e36013194b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:29.740734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 346/683] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 22 11:01:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:29.740734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ad63b-478c-4c02-8d65-72e36013194b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:29.766254 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:29.806736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:29.818816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 338/684] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:29.820225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed334bc9-ce31-47d2-b02a-1c57d42bc19f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:29.872067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ad63b-478c-4c02-8d65-72e36013194b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:29.916562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929e620-6703-4c9b-b16f-e529f45e12c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:29.918216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929e620-6703-4c9b-b16f-e529f45e12c8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.918715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929e620-6703-4c9b-b16f-e529f45e12c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.919232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929e620-6703-4c9b-b16f-e529f45e12c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:29.921213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929e620-6703-4c9b-b16f-e529f45e12c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:29.921492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929e620-6703-4c9b-b16f-e529f45e12c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:29.922977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929e620-6703-4c9b-b16f-e529f45e12c8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:29.938049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 347/685] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:29.939146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929e620-6703-4c9b-b16f-e529f45e12c8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:29.941522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed334bc9-ce31-47d2-b02a-1c57d42bc19f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:29.989194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8eda7-6814-41a9-aae2-9bee429cc47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:29.990218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8eda7-6814-41a9-aae2-9bee429cc47d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:29.990762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8eda7-6814-41a9-aae2-9bee429cc47d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:29.991251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8eda7-6814-41a9-aae2-9bee429cc47d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:30.037937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:30.056528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929e620-6703-4c9b-b16f-e529f45e12c8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:30.071241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8eda7-6814-41a9-aae2-9bee429cc47d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:30.071753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:30.074714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8eda7-6814-41a9-aae2-9bee429cc47d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:30.097035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88aded1-d394-4889-8027-8f1684d8bb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:30.097893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88aded1-d394-4889-8027-8f1684d8bb86 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:30.098177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88aded1-d394-4889-8027-8f1684d8bb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:30.098452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88aded1-d394-4889-8027-8f1684d8bb86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:30.111324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88aded1-d394-4889-8027-8f1684d8bb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:30.112759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88aded1-d394-4889-8027-8f1684d8bb86 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e88aded1-d394-4889-8027-8f1684d8bb86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:30.130947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:30.131978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:30.132909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:30.132909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:30.132909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:30.132909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:30.132909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 348/686] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:30.133634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88aded1-d394-4889-8027-8f1684d8bb86 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:30.197935 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 339/687] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:30.203925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8eda7-6814-41a9-aae2-9bee429cc47d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:30.254188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88aded1-d394-4889-8027-8f1684d8bb86 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:30.298432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218397e-ae2b-4488-a675-39cf50e88f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:30.299138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218397e-ae2b-4488-a675-39cf50e88f91 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:30.299457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218397e-ae2b-4488-a675-39cf50e88f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:30.299700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:30.299855 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218397e-ae2b-4488-a675-39cf50e88f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:30.300650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218397e-ae2b-4488-a675-39cf50e88f91 None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=5b04cb4117974f8eb9bbf3a41c48a11a) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:30.301015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218397e-ae2b-4488-a675-39cf50e88f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:30.302732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218397e-ae2b-4488-a675-39cf50e88f91 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:30.316011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 349/688] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:30 2022] GET /identity/v3/role_assignments?user.id=5b04cb4117974f8eb9bbf3a41c48a11a&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:30.316777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218397e-ae2b-4488-a675-39cf50e88f91 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:30.335304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8eda7-6814-41a9-aae2-9bee429cc47d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:30.387614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:30.413485 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e6532-8acd-4eae-9c06-173c4c0def7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:30.414173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e6532-8acd-4eae-9c06-173c4c0def7c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:30.414461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e6532-8acd-4eae-9c06-173c4c0def7c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:30.414737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e6532-8acd-4eae-9c06-173c4c0def7c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:30.415525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e6532-8acd-4eae-9c06-173c4c0def7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:30.415879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e6532-8acd-4eae-9c06-173c4c0def7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:30.417362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e6532-8acd-4eae-9c06-173c4c0def7c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:30.446513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 340/689] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:30.450935 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e6532-8acd-4eae-9c06-173c4c0def7c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:30.513009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218397e-ae2b-4488-a675-39cf50e88f91 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:30.557548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e532ec-5f32-48ee-81e7-65c27772362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:30.558367 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e532ec-5f32-48ee-81e7-65c27772362d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:30.558817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e532ec-5f32-48ee-81e7-65c27772362d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:30.559104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e532ec-5f32-48ee-81e7-65c27772362d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:30.575663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e532ec-5f32-48ee-81e7-65c27772362d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:30.578098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e532ec-5f32-48ee-81e7-65c27772362d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d2e532ec-5f32-48ee-81e7-65c27772362d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:30.600677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:30.601566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:30.602475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:30.602475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:30.602475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:30.602475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:30.603634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 350/690] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:30.608026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e532ec-5f32-48ee-81e7-65c27772362d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:30.632645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e6532-8acd-4eae-9c06-173c4c0def7c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:30.669949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:30.677446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fc836-62e3-4707-982a-75103efe70ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:30.678107 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fc836-62e3-4707-982a-75103efe70ec None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:30.678400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fc836-62e3-4707-982a-75103efe70ec None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:30.678681 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fc836-62e3-4707-982a-75103efe70ec None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:30.679431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fc836-62e3-4707-982a-75103efe70ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:30.680898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fc836-62e3-4707-982a-75103efe70ec None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:30.710970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 341/691] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:01:30 2022] POST /identity/v3/users => generated 254 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:01:30.722265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:30.745761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e532ec-5f32-48ee-81e7-65c27772362d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:30.786110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457e74a-4585-4ba3-8508-4e964a7bc136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:30.786853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457e74a-4585-4ba3-8508-4e964a7bc136 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:30.787141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457e74a-4585-4ba3-8508-4e964a7bc136 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:30.787492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457e74a-4585-4ba3-8508-4e964a7bc136 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:30.788239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457e74a-4585-4ba3-8508-4e964a7bc136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:30.788550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457e74a-4585-4ba3-8508-4e964a7bc136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:30.789967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457e74a-4585-4ba3-8508-4e964a7bc136 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:30.806142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 351/692] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:30.812098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fc836-62e3-4707-982a-75103efe70ec None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:30.912755 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99a208-3eed-4d08-83ea-1097e8e75dc2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:30.913162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99a208-3eed-4d08-83ea-1097e8e75dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:30.913517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99a208-3eed-4d08-83ea-1097e8e75dc2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:30.915253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 352/693] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:30.921921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c1366-5dd5-423d-a528-b609651c18e6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:30.923496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c1366-5dd5-423d-a528-b609651c18e6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:30.924320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c1366-5dd5-423d-a528-b609651c18e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:30.948628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c79c1366-5dd5-423d-a528-b609651c18e6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:30.950180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c79c1366-5dd5-423d-a528-b609651c18e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:30.987623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fc836-62e3-4707-982a-75103efe70ec None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:31.030740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c835de-1cdb-4e7a-a5f2-efac784f52a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:31.031624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c835de-1cdb-4e7a-a5f2-efac784f52a2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:31.031911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c835de-1cdb-4e7a-a5f2-efac784f52a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:31.032188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c835de-1cdb-4e7a-a5f2-efac784f52a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:31.040490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c835de-1cdb-4e7a-a5f2-efac784f52a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:31.041683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 353/694] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:31.042603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c835de-1cdb-4e7a-a5f2-efac784f52a2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-53c835de-1cdb-4e7a-a5f2-efac784f52a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:31.058696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:31.059708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:31.060626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:31.060626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:31.060626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:31.060626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:31.060626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:31.060626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:31.060626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:31.060626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:31.060626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 342/695] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:31.066300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c1366-5dd5-423d-a528-b609651c18e6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:31.073011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c835de-1cdb-4e7a-a5f2-efac784f52a2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:31.127806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:31.155216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:31.190172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c835de-1cdb-4e7a-a5f2-efac784f52a2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:31.190370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c1366-5dd5-423d-a528-b609651c18e6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:31.228999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3de611-a893-4daf-bb50-0dc33ad033d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:31.229825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3de611-a893-4daf-bb50-0dc33ad033d4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:31.230105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3de611-a893-4daf-bb50-0dc33ad033d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:31.230376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3de611-a893-4daf-bb50-0dc33ad033d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:31.232884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a77338-e7b6-482a-b696-b03308e9e098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:31.233739 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a77338-e7b6-482a-b696-b03308e9e098 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:31.234036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a77338-e7b6-482a-b696-b03308e9e098 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:31.235421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a77338-e7b6-482a-b696-b03308e9e098 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:31.235421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a77338-e7b6-482a-b696-b03308e9e098 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:31.235821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a77338-e7b6-482a-b696-b03308e9e098 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:31.236967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a77338-e7b6-482a-b696-b03308e9e098 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:31.244630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3de611-a893-4daf-bb50-0dc33ad033d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:31.253692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 354/696] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-fa3de611-a893-4daf-bb50-0dc33ad033d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:31.257517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:31.258518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:31.259400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 343/697] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:31.268677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a77338-e7b6-482a-b696-b03308e9e098 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:31.269221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3de611-a893-4daf-bb50-0dc33ad033d4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:31.458198 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3de611-a893-4daf-bb50-0dc33ad033d4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:31.473711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:31.490546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a77338-e7b6-482a-b696-b03308e9e098 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:31.526620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb6f4d-ef99-40b1-9852-2c1d7d49a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:31.527330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb6f4d-ef99-40b1-9852-2c1d7d49a75d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:31.527621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb6f4d-ef99-40b1-9852-2c1d7d49a75d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:31.527900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb6f4d-ef99-40b1-9852-2c1d7d49a75d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:31.528750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb6f4d-ef99-40b1-9852-2c1d7d49a75d None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=e7935e85cc3f4ed1ad5b01ba892e6e82) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:31.529068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb6f4d-ef99-40b1-9852-2c1d7d49a75d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:31.530534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb6f4d-ef99-40b1-9852-2c1d7d49a75d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:31.539018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 344/698] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:31 2022] GET /identity/v3/role_assignments?user.id=e7935e85cc3f4ed1ad5b01ba892e6e82&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:31.547784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:31.556254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952065c-5eff-468c-95c1-4495f8c70ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:31.557352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952065c-5eff-468c-95c1-4495f8c70ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:31.557741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952065c-5eff-468c-95c1-4495f8c70ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:31.558204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952065c-5eff-468c-95c1-4495f8c70ac5 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:31.559329 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952065c-5eff-468c-95c1-4495f8c70ac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:31.559869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952065c-5eff-468c-95c1-4495f8c70ac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:31.561518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952065c-5eff-468c-95c1-4495f8c70ac5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:31.577989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 355/699] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:31.583804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb6f4d-ef99-40b1-9852-2c1d7d49a75d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:31.730821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb6f4d-ef99-40b1-9852-2c1d7d49a75d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:31.785225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d62610-608d-40d4-897b-2d9d36177141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:31.786044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d62610-608d-40d4-897b-2d9d36177141 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:31.786335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d62610-608d-40d4-897b-2d9d36177141 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:31.786607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d62610-608d-40d4-897b-2d9d36177141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:31.803280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d62610-608d-40d4-897b-2d9d36177141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:31.804788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d62610-608d-40d4-897b-2d9d36177141 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a1d62610-608d-40d4-897b-2d9d36177141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:31.824235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:31.825854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:31.827512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:31.827512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:31.827512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:31.827512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:31.827512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 345/700] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:31.831643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952065c-5eff-468c-95c1-4495f8c70ac5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:31.985705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952065c-5eff-468c-95c1-4495f8c70ac5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:32.029440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414dc741-c34d-44dd-8bcd-230663f36c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:32.030148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dc741-c34d-44dd-8bcd-230663f36c90 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:32.030440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dc741-c34d-44dd-8bcd-230663f36c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:32.030724 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dc741-c34d-44dd-8bcd-230663f36c90 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:32.031512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414dc741-c34d-44dd-8bcd-230663f36c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:32.031824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414dc741-c34d-44dd-8bcd-230663f36c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:32.033273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414dc741-c34d-44dd-8bcd-230663f36c90 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:32.047628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 356/701] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:32.053198 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d62610-608d-40d4-897b-2d9d36177141 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:32.086476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:32.151747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0ae23-f148-46fa-9883-335bc3687b72 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:32.152069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0ae23-f148-46fa-9883-335bc3687b72 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:32.152341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0ae23-f148-46fa-9883-335bc3687b72 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:32.154040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 357/702] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:32.161204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c05ed-2189-484c-a039-2dcbed2364e3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:32.161485 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c05ed-2189-484c-a039-2dcbed2364e3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:32.161772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c05ed-2189-484c-a039-2dcbed2364e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:32.180457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9a5c05ed-2189-484c-a039-2dcbed2364e3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:32.182218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9a5c05ed-2189-484c-a039-2dcbed2364e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:32.187079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d62610-608d-40d4-897b-2d9d36177141 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:32.231062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa458f3b-0410-4b20-aac9-20834a386dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:32.232081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa458f3b-0410-4b20-aac9-20834a386dcc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:32.232377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa458f3b-0410-4b20-aac9-20834a386dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:32.232662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa458f3b-0410-4b20-aac9-20834a386dcc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:32.238728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:32.252689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa458f3b-0410-4b20-aac9-20834a386dcc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:32.254264 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa458f3b-0410-4b20-aac9-20834a386dcc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:32.261189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 358/703] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-aa458f3b-0410-4b20-aac9-20834a386dcc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:32.275756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:32.276778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:32.278139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 346/704] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 22 11:01:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:32.283670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c05ed-2189-484c-a039-2dcbed2364e3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:32.292077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa458f3b-0410-4b20-aac9-20834a386dcc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:32.388442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:32.406964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa458f3b-0410-4b20-aac9-20834a386dcc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:32.446807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c05ed-2189-484c-a039-2dcbed2364e3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:32.483275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee6d57-a711-4b72-a6e3-0827b5a801c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:32.484097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee6d57-a711-4b72-a6e3-0827b5a801c8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:32.484377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee6d57-a711-4b72-a6e3-0827b5a801c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:32.484647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee6d57-a711-4b72-a6e3-0827b5a801c8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:32.485306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee6d57-a711-4b72-a6e3-0827b5a801c8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:32.486677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee6d57-a711-4b72-a6e3-0827b5a801c8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:32.488916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d268554-32a0-4db9-a1b6-0f6c87fc2b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:32.489537 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d268554-32a0-4db9-a1b6-0f6c87fc2b88 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:32.489830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d268554-32a0-4db9-a1b6-0f6c87fc2b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:32.490092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d268554-32a0-4db9-a1b6-0f6c87fc2b88 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:32.491003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d268554-32a0-4db9-a1b6-0f6c87fc2b88 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:32.491334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d268554-32a0-4db9-a1b6-0f6c87fc2b88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:32.492811 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d268554-32a0-4db9-a1b6-0f6c87fc2b88 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c5ee6d57-a711-4b72-a6e3-0827b5a801c8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:32.514494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:32.515687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:32.516567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 347/705] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:32 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:32.526647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee6d57-a711-4b72-a6e3-0827b5a801c8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:32.533021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 359/706] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 22 11:01:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:32.539651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d268554-32a0-4db9-a1b6-0f6c87fc2b88 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:32.597282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:32.661069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d268554-32a0-4db9-a1b6-0f6c87fc2b88 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:32.702929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f790fcf-ef78-44f8-bd0c-ca6f432ee2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:32.706279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f790fcf-ef78-44f8-bd0c-ca6f432ee2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:32.706577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f790fcf-ef78-44f8-bd0c-ca6f432ee2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:32.706862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f790fcf-ef78-44f8-bd0c-ca6f432ee2f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:32.770718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee6d57-a711-4b72-a6e3-0827b5a801c8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:32.835292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f790fcf-ef78-44f8-bd0c-ca6f432ee2f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:32.835524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9722ec7-d8dc-406c-98c8-10b684be2462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:32.835524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9722ec7-d8dc-406c-98c8-10b684be2462 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:32.835524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9722ec7-d8dc-406c-98c8-10b684be2462 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:32.835524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9722ec7-d8dc-406c-98c8-10b684be2462 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:32.835793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9722ec7-d8dc-406c-98c8-10b684be2462 None admin] RBAC: Adding query filter params (name=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:32.835793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9722ec7-d8dc-406c-98c8-10b684be2462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:32.835793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9722ec7-d8dc-406c-98c8-10b684be2462 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:32.839233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f790fcf-ef78-44f8-bd0c-ca6f432ee2f6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:32.841226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 348/707] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:01:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:32.851323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9722ec7-d8dc-406c-98c8-10b684be2462 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:32.876130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:32.929307 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:33.034457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9722ec7-d8dc-406c-98c8-10b684be2462 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:33.057482 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 360/708] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:33.060332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f790fcf-ef78-44f8-bd0c-ca6f432ee2f6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:33.108428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07b399-3fb0-4045-b1c6-391e8bdd5080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:33.109744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07b399-3fb0-4045-b1c6-391e8bdd5080 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.110190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07b399-3fb0-4045-b1c6-391e8bdd5080 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.110624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07b399-3fb0-4045-b1c6-391e8bdd5080 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.112202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07b399-3fb0-4045-b1c6-391e8bdd5080 None admin] RBAC: Adding query filter params (type=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:33.112683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07b399-3fb0-4045-b1c6-391e8bdd5080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:33.114796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07b399-3fb0-4045-b1c6-391e8bdd5080 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:33.122791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 349/709] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:01:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:33.217075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f790fcf-ef78-44f8-bd0c-ca6f432ee2f6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:33.259159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703e6b6-6d74-4411-9fde-becd4a66b252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:33.260879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703e6b6-6d74-4411-9fde-becd4a66b252 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.261354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703e6b6-6d74-4411-9fde-becd4a66b252 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.261788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703e6b6-6d74-4411-9fde-becd4a66b252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.274731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703e6b6-6d74-4411-9fde-becd4a66b252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:33.277276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703e6b6-6d74-4411-9fde-becd4a66b252 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:33.279716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9703e6b6-6d74-4411-9fde-becd4a66b252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:33.304733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:33.305964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:33.307182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:33.307182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:33.307182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:33.307182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:33.309455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 361/710] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:33.314831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07b399-3fb0-4045-b1c6-391e8bdd5080 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:33.368313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55fa9b-17fa-4765-81f1-0692134c8617 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.368609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55fa9b-17fa-4765-81f1-0692134c8617 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.368944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55fa9b-17fa-4765-81f1-0692134c8617 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.370723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 362/711] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:33.376426 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941577d-be2d-4c71-9ef9-934e73ef29ca None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.376730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941577d-be2d-4c71-9ef9-934e73ef29ca None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.377021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941577d-be2d-4c71-9ef9-934e73ef29ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.401355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5941577d-be2d-4c71-9ef9-934e73ef29ca None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:33.402743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5941577d-be2d-4c71-9ef9-934e73ef29ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:33.428725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:33.457994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07b399-3fb0-4045-b1c6-391e8bdd5080 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:33.507055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 363/712] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:01:33.507055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab71b62-e27f-4a4b-9b48-a8d3a7920de0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.507055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab71b62-e27f-4a4b-9b48-a8d3a7920de0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.507055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab71b62-e27f-4a4b-9b48-a8d3a7920de0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.507055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 364/713] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:33.513360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0615bd-d04e-4554-b7aa-44c388956eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:33.514047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0615bd-d04e-4554-b7aa-44c388956eff None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.514335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0615bd-d04e-4554-b7aa-44c388956eff None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.514613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0615bd-d04e-4554-b7aa-44c388956eff None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.515399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0615bd-d04e-4554-b7aa-44c388956eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:33.515714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0615bd-d04e-4554-b7aa-44c388956eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:33.517100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0615bd-d04e-4554-b7aa-44c388956eff None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:33.519049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95706cdf-634b-4dbc-a766-2b830584abf6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.519370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95706cdf-634b-4dbc-a766-2b830584abf6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.519649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95706cdf-634b-4dbc-a766-2b830584abf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.543244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-95706cdf-634b-4dbc-a766-2b830584abf6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:33.545038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 350/714] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:33.545257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-95706cdf-634b-4dbc-a766-2b830584abf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:33.545896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0615bd-d04e-4554-b7aa-44c388956eff None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:33.643288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 365/715] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:33.643288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95706cdf-634b-4dbc-a766-2b830584abf6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:33.701355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0615bd-d04e-4554-b7aa-44c388956eff None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:33.707045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:33.741921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7e14f-e2e3-47d0-b878-681a753d0438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:33.743244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7e14f-e2e3-47d0-b878-681a753d0438 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.744322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7e14f-e2e3-47d0-b878-681a753d0438 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.744822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7e14f-e2e3-47d0-b878-681a753d0438 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.750252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:33.760869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7e14f-e2e3-47d0-b878-681a753d0438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:33.764873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95706cdf-634b-4dbc-a766-2b830584abf6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5be7e14f-e2e3-47d0-b878-681a753d0438 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:33.769692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:33.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:33.771624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:33.779345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 351/716] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:33.780667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7e14f-e2e3-47d0-b878-681a753d0438 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:33.803940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695507b-9dce-4cb6-a2c9-360143122f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:33.804796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695507b-9dce-4cb6-a2c9-360143122f1e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.805088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695507b-9dce-4cb6-a2c9-360143122f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.805367 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695507b-9dce-4cb6-a2c9-360143122f1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.813912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695507b-9dce-4cb6-a2c9-360143122f1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:33.815548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695507b-9dce-4cb6-a2c9-360143122f1e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b695507b-9dce-4cb6-a2c9-360143122f1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:33.831124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:33.832139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 366/717] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:33.833692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695507b-9dce-4cb6-a2c9-360143122f1e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:33.900120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7e14f-e2e3-47d0-b878-681a753d0438 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:33.938789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529d7f0-3471-4c2b-8b31-e9cb77d7d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:33.940021 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529d7f0-3471-4c2b-8b31-e9cb77d7d23e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:33.940549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529d7f0-3471-4c2b-8b31-e9cb77d7d23e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:33.941055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529d7f0-3471-4c2b-8b31-e9cb77d7d23e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:33.941940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529d7f0-3471-4c2b-8b31-e9cb77d7d23e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:33.943629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529d7f0-3471-4c2b-8b31-e9cb77d7d23e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:33.948613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695507b-9dce-4cb6-a2c9-360143122f1e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3529d7f0-3471-4c2b-8b31-e9cb77d7d23e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:33.957743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:33.960275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 22 11:01:33.961818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:33.972275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 352/718] 10.222.0.20 () {56 vars in 1129 bytes} [Fri Jul 22 11:01:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:33.973638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529d7f0-3471-4c2b-8b31-e9cb77d7d23e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:33.989375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:33.999671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c697ebf-e068-4d63-ba2e-4d658335e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:34.000413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c697ebf-e068-4d63-ba2e-4d658335e5db None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.000702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c697ebf-e068-4d63-ba2e-4d658335e5db None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.000991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c697ebf-e068-4d63-ba2e-4d658335e5db None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.001778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c697ebf-e068-4d63-ba2e-4d658335e5db None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:34.002084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c697ebf-e068-4d63-ba2e-4d658335e5db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:34.004096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c697ebf-e068-4d63-ba2e-4d658335e5db None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:34.018100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 367/719] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:34.018100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c697ebf-e068-4d63-ba2e-4d658335e5db None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:34.080275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:34.127401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529d7f0-3471-4c2b-8b31-e9cb77d7d23e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:34.146491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c697ebf-e068-4d63-ba2e-4d658335e5db None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:34.166255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d43ff-4966-46b8-b24d-fd2599666ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:34.167234 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d43ff-4966-46b8-b24d-fd2599666ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.167740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d43ff-4966-46b8-b24d-fd2599666ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.168200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d43ff-4966-46b8-b24d-fd2599666ea4 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.169210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d43ff-4966-46b8-b24d-fd2599666ea4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:34.169726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d43ff-4966-46b8-b24d-fd2599666ea4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:34.171395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d43ff-4966-46b8-b24d-fd2599666ea4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:34.189804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 353/720] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:34.189804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d43ff-4966-46b8-b24d-fd2599666ea4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:34.190786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6e3f2-09ad-4462-b9dc-33e82810319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:34.191236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6e3f2-09ad-4462-b9dc-33e82810319f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.191554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6e3f2-09ad-4462-b9dc-33e82810319f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.191908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6e3f2-09ad-4462-b9dc-33e82810319f None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.192702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6e3f2-09ad-4462-b9dc-33e82810319f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:34.193009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6e3f2-09ad-4462-b9dc-33e82810319f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:34.194530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6e3f2-09ad-4462-b9dc-33e82810319f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:34.205484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 368/721] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 22 11:01:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:34.205484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6e3f2-09ad-4462-b9dc-33e82810319f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:34.248504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:34.309094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d43ff-4966-46b8-b24d-fd2599666ea4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:34.316518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6e3f2-09ad-4462-b9dc-33e82810319f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:34.332473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:34.348061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14d0b1-c887-482c-9d13-4885aa6d3089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:34.349219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14d0b1-c887-482c-9d13-4885aa6d3089 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.349715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14d0b1-c887-482c-9d13-4885aa6d3089 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.350219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14d0b1-c887-482c-9d13-4885aa6d3089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.358313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94da439-257b-4fa8-ac96-bb43ece57b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:34.358635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94da439-257b-4fa8-ac96-bb43ece57b1a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.358922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94da439-257b-4fa8-ac96-bb43ece57b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.359207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94da439-257b-4fa8-ac96-bb43ece57b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.360064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94da439-257b-4fa8-ac96-bb43ece57b1a None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=89224d740c7148938c2d803804083a57, user.id=c51929d0ecb047b4bdc405dc86fe7eb6) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:34.360380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94da439-257b-4fa8-ac96-bb43ece57b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:34.361928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94da439-257b-4fa8-ac96-bb43ece57b1a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:34.364114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14d0b1-c887-482c-9d13-4885aa6d3089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:34.366121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14d0b1-c887-482c-9d13-4885aa6d3089 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:34.375813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 369/722] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:34 2022] GET /identity/v3/role_assignments?user.id=c51929d0ecb047b4bdc405dc86fe7eb6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:34.376829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94da439-257b-4fa8-ac96-bb43ece57b1a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5c14d0b1-c887-482c-9d13-4885aa6d3089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:34.388067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:34.389020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:34.389921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:34.389921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:34.389921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:34.389921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:34.395391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 354/723] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:34.398647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14d0b1-c887-482c-9d13-4885aa6d3089 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:34.519436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14d0b1-c887-482c-9d13-4885aa6d3089 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:34.530415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94da439-257b-4fa8-ac96-bb43ece57b1a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:34.533298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:34.560558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392faf7-3cc9-4d0a-b0d7-5504fe9dfcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:34.561638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392faf7-3cc9-4d0a-b0d7-5504fe9dfcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.562173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392faf7-3cc9-4d0a-b0d7-5504fe9dfcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.562687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392faf7-3cc9-4d0a-b0d7-5504fe9dfcc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.563714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392faf7-3cc9-4d0a-b0d7-5504fe9dfcc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:34.564252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392faf7-3cc9-4d0a-b0d7-5504fe9dfcc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:34.565877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392faf7-3cc9-4d0a-b0d7-5504fe9dfcc7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:34.572983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77649827-8ec3-48d1-a701-4a9b83e33b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:34.573736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77649827-8ec3-48d1-a701-4a9b83e33b02 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.574024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77649827-8ec3-48d1-a701-4a9b83e33b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.574306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77649827-8ec3-48d1-a701-4a9b83e33b02 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.575034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77649827-8ec3-48d1-a701-4a9b83e33b02 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:34.575416 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77649827-8ec3-48d1-a701-4a9b83e33b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:34.576947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77649827-8ec3-48d1-a701-4a9b83e33b02 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:34.586450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 370/725] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 22 11:01:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:34.586450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977272e-8a82-40b0-b982-e158e398460b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.586450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977272e-8a82-40b0-b982-e158e398460b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.586450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977272e-8a82-40b0-b982-e158e398460b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.587060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 355/725] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:34.588849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392faf7-3cc9-4d0a-b0d7-5504fe9dfcc7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:34.591563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 371/726] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:34.604688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411acd85-f14b-4a3b-a066-224188a70f82 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.605057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411acd85-f14b-4a3b-a066-224188a70f82 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.605279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411acd85-f14b-4a3b-a066-224188a70f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.625330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-411acd85-f14b-4a3b-a066-224188a70f82 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:34.626700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-411acd85-f14b-4a3b-a066-224188a70f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:34.632298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:34.717206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 372/727] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:34.726604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392faf7-3cc9-4d0a-b0d7-5504fe9dfcc7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:34.781788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411acd85-f14b-4a3b-a066-224188a70f82 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:34.789574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c81d0-1eea-4165-84e6-743e79dc5c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:34.790519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c81d0-1eea-4165-84e6-743e79dc5c05 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:34.791359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c81d0-1eea-4165-84e6-743e79dc5c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:34.791640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c81d0-1eea-4165-84e6-743e79dc5c05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:34.803660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c81d0-1eea-4165-84e6-743e79dc5c05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:34.805544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c81d0-1eea-4165-84e6-743e79dc5c05 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-532c81d0-1eea-4165-84e6-743e79dc5c05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:34.826178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:34.827181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:34.828345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 356/728] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:01:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:34.835832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c81d0-1eea-4165-84e6-743e79dc5c05 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:34.916229 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:34.924979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411acd85-f14b-4a3b-a066-224188a70f82 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:34.979016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:34.997603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c81d0-1eea-4165-84e6-743e79dc5c05 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:35.004870 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05192068-aaa8-486c-afba-dcd0c9da75d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:35.006069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05192068-aaa8-486c-afba-dcd0c9da75d4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.006567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05192068-aaa8-486c-afba-dcd0c9da75d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.007047 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05192068-aaa8-486c-afba-dcd0c9da75d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.032345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05192068-aaa8-486c-afba-dcd0c9da75d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:35.036605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b77a5-62d5-4ed5-a237-16f19bb41e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:35.037287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b77a5-62d5-4ed5-a237-16f19bb41e57 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.037575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b77a5-62d5-4ed5-a237-16f19bb41e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.037856 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b77a5-62d5-4ed5-a237-16f19bb41e57 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.038710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b77a5-62d5-4ed5-a237-16f19bb41e57 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:35.039025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b77a5-62d5-4ed5-a237-16f19bb41e57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:35.040636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b77a5-62d5-4ed5-a237-16f19bb41e57 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-05192068-aaa8-486c-afba-dcd0c9da75d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:35.052044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:35.053046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:35.053889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:35.056875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 373/729] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:35.063592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05192068-aaa8-486c-afba-dcd0c9da75d4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:35.064963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 357/730] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 22 11:01:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:35.071573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b77a5-62d5-4ed5-a237-16f19bb41e57 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:35.182967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b77a5-62d5-4ed5-a237-16f19bb41e57 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:35.221171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af841b-7a02-45a6-8475-3a7921f87707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:35.221832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af841b-7a02-45a6-8475-3a7921f87707 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.222124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af841b-7a02-45a6-8475-3a7921f87707 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.222404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af841b-7a02-45a6-8475-3a7921f87707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.247669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:35.300243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af841b-7a02-45a6-8475-3a7921f87707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:35.301761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af841b-7a02-45a6-8475-3a7921f87707 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:35.320014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05192068-aaa8-486c-afba-dcd0c9da75d4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:35.399497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337a5c2-e215-43fb-9778-86b12be0b5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:35.401080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337a5c2-e215-43fb-9778-86b12be0b5ec None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.401612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337a5c2-e215-43fb-9778-86b12be0b5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.402096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337a5c2-e215-43fb-9778-86b12be0b5ec None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.411289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337a5c2-e215-43fb-9778-86b12be0b5ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:35.411887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337a5c2-e215-43fb-9778-86b12be0b5ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:35.413480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337a5c2-e215-43fb-9778-86b12be0b5ec None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:35.429754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 374/731] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:35.440251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337a5c2-e215-43fb-9778-86b12be0b5ec None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:35.458879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 358/732] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:35.464975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af841b-7a02-45a6-8475-3a7921f87707 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:35.497876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:35.526527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:35.579074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337a5c2-e215-43fb-9778-86b12be0b5ec None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:35.589342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af841b-7a02-45a6-8475-3a7921f87707 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:35.614688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f0f87-95ce-4ab2-9216-952d6e19c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:35.615626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f0f87-95ce-4ab2-9216-952d6e19c988 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.615998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f0f87-95ce-4ab2-9216-952d6e19c988 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.616346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f0f87-95ce-4ab2-9216-952d6e19c988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.627452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f0f87-95ce-4ab2-9216-952d6e19c988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:35.629005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f0f87-95ce-4ab2-9216-952d6e19c988 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:35.634868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ead62-3166-4c68-ac0e-105144d484af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:35.636092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ead62-3166-4c68-ac0e-105144d484af None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.636590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ead62-3166-4c68-ac0e-105144d484af None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.637054 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ead62-3166-4c68-ac0e-105144d484af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fe0f0f87-95ce-4ab2-9216-952d6e19c988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:35.646884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:35.647838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:35.648756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:35.648756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:35.648756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:35.648756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:35.650648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ead62-3166-4c68-ac0e-105144d484af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:35.652469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ead62-3166-4c68-ac0e-105144d484af None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:35.653283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 375/733] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:35.654591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42823c2-5482-4992-a367-8c169f252b5b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.654890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42823c2-5482-4992-a367-8c169f252b5b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.655179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42823c2-5482-4992-a367-8c169f252b5b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.657066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 376/734] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:35.657830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42823c2-5482-4992-a367-8c169f252b5b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d11ead62-3166-4c68-ac0e-105144d484af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:35.673276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:35.674407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:35.675379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:35.675379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:35.675379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:35.675379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:35.677307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 359/735] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:35.678661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767dff3-13d8-4eb9-9232-4e4491b6d196 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.679098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767dff3-13d8-4eb9-9232-4e4491b6d196 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.679560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767dff3-13d8-4eb9-9232-4e4491b6d196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.699534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3767dff3-13d8-4eb9-9232-4e4491b6d196 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:35.701157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3767dff3-13d8-4eb9-9232-4e4491b6d196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:35.740930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:35.779801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42823c2-5482-4992-a367-8c169f252b5b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:35.800551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 360/736] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:35.801712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767dff3-13d8-4eb9-9232-4e4491b6d196 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:35.837369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253adcf0-0f41-4c8f-ac49-a9a897d4bcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:35.838002 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253adcf0-0f41-4c8f-ac49-a9a897d4bcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.838335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253adcf0-0f41-4c8f-ac49-a9a897d4bcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.838664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253adcf0-0f41-4c8f-ac49-a9a897d4bcb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.838891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:35.839628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253adcf0-0f41-4c8f-ac49-a9a897d4bcb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:35.839802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253adcf0-0f41-4c8f-ac49-a9a897d4bcb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:35.841234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253adcf0-0f41-4c8f-ac49-a9a897d4bcb4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:35.920863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767dff3-13d8-4eb9-9232-4e4491b6d196 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:35.965644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8777a9-8a86-4ca0-a17b-cf62800e4365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:35.967115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8777a9-8a86-4ca0-a17b-cf62800e4365 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:35.967746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8777a9-8a86-4ca0-a17b-cf62800e4365 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:35.968286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8777a9-8a86-4ca0-a17b-cf62800e4365 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:35.969177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8777a9-8a86-4ca0-a17b-cf62800e4365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:35.969676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8777a9-8a86-4ca0-a17b-cf62800e4365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:35.974519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 377/737] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:35.974519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253adcf0-0f41-4c8f-ac49-a9a897d4bcb4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:35.976980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8777a9-8a86-4ca0-a17b-cf62800e4365 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:36.000219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 361/738] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:36.002171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8777a9-8a86-4ca0-a17b-cf62800e4365 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:36.084493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253adcf0-0f41-4c8f-ac49-a9a897d4bcb4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:36.118585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:36.126832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0c147-149b-4847-9bcf-0003cfa777c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:36.128088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0c147-149b-4847-9bcf-0003cfa777c3 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:36.128532 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0c147-149b-4847-9bcf-0003cfa777c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:36.128989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0c147-149b-4847-9bcf-0003cfa777c3 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:36.130136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0c147-149b-4847-9bcf-0003cfa777c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:36.131650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:36.133314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0c147-149b-4847-9bcf-0003cfa777c3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:36.143025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8777a9-8a86-4ca0-a17b-cf62800e4365 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:36.155432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 378/739] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:01:35 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:36.156651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0c147-149b-4847-9bcf-0003cfa777c3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:36.234681 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f11a9-071e-4fd4-a703-de0af1ea5e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:36.237496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f11a9-071e-4fd4-a703-de0af1ea5e38 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:36.238194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f11a9-071e-4fd4-a703-de0af1ea5e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:36.239670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f11a9-071e-4fd4-a703-de0af1ea5e38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:36.262132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f11a9-071e-4fd4-a703-de0af1ea5e38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:36.263646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f11a9-071e-4fd4-a703-de0af1ea5e38 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-443f11a9-071e-4fd4-a703-de0af1ea5e38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:36.287319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:36.288368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:36.289622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 362/740] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:36.296860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f11a9-071e-4fd4-a703-de0af1ea5e38 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:36.335687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0c147-149b-4847-9bcf-0003cfa777c3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:36.386196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800ef73-ca10-4b23-9161-e98e6ddab76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:36.388161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800ef73-ca10-4b23-9161-e98e6ddab76d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:36.388737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800ef73-ca10-4b23-9161-e98e6ddab76d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:36.389171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800ef73-ca10-4b23-9161-e98e6ddab76d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:36.399834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800ef73-ca10-4b23-9161-e98e6ddab76d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:36.401808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800ef73-ca10-4b23-9161-e98e6ddab76d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1800ef73-ca10-4b23-9161-e98e6ddab76d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:36.422631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:36.423735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:36.425005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:36.425005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:36.425005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:36.425005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:36.425005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:36.425005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:36.425005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:36.425005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:36.432365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 379/741] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:36.436017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800ef73-ca10-4b23-9161-e98e6ddab76d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:36.447037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f11a9-071e-4fd4-a703-de0af1ea5e38 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:36.493079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb53cc0-cc1a-41c1-bde5-547ea1745dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:36.494236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb53cc0-cc1a-41c1-bde5-547ea1745dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:36.494552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb53cc0-cc1a-41c1-bde5-547ea1745dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:36.494836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb53cc0-cc1a-41c1-bde5-547ea1745dd5 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:36.495729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb53cc0-cc1a-41c1-bde5-547ea1745dd5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:36.496043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb53cc0-cc1a-41c1-bde5-547ea1745dd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:36.498222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb53cc0-cc1a-41c1-bde5-547ea1745dd5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:36.507292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:36.527293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 363/742] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 22 11:01:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:36.532704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb53cc0-cc1a-41c1-bde5-547ea1745dd5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:36.545444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:36.579838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800ef73-ca10-4b23-9161-e98e6ddab76d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:36.651872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4394a-fb01-4e13-9e59-507b8db3f884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:36.653730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4394a-fb01-4e13-9e59-507b8db3f884 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:36.654506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4394a-fb01-4e13-9e59-507b8db3f884 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:36.655161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4394a-fb01-4e13-9e59-507b8db3f884 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:36.657346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4394a-fb01-4e13-9e59-507b8db3f884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:36.658076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4394a-fb01-4e13-9e59-507b8db3f884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:36.660577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4394a-fb01-4e13-9e59-507b8db3f884 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:36.696490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 380/743] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:36.701120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4394a-fb01-4e13-9e59-507b8db3f884 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:36.725069 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb53cc0-cc1a-41c1-bde5-547ea1745dd5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:36.821369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c5c2f-f17b-4455-a8a5-89737eb6f750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:36.822559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c5c2f-f17b-4455-a8a5-89737eb6f750 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:36.822851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c5c2f-f17b-4455-a8a5-89737eb6f750 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:36.823228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c5c2f-f17b-4455-a8a5-89737eb6f750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:36.828201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4394a-fb01-4e13-9e59-507b8db3f884 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:36.842883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:36.873266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c68d3-f3da-48a4-899e-a75245ac60d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:36.874497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c68d3-f3da-48a4-899e-a75245ac60d9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:36.874884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c68d3-f3da-48a4-899e-a75245ac60d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:36.875243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c68d3-f3da-48a4-899e-a75245ac60d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:36.876251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c68d3-f3da-48a4-899e-a75245ac60d9 None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=646d7c5e6c8e4825a7fe06d018ee319c) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:36.876696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c68d3-f3da-48a4-899e-a75245ac60d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:36.878475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c68d3-f3da-48a4-899e-a75245ac60d9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:36.886580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 381/744] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:36 2022] GET /identity/v3/role_assignments?user.id=646d7c5e6c8e4825a7fe06d018ee319c&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:36.892802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b3e03-6346-43e7-bf51-67862e26d5c9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:36.893104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b3e03-6346-43e7-bf51-67862e26d5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:36.893367 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b3e03-6346-43e7-bf51-67862e26d5c9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:36.899726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 382/745] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:36.911335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65f32-2c2b-4d33-9c2e-5e6e025b0979 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:36.911623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65f32-2c2b-4d33-9c2e-5e6e025b0979 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:36.911863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65f32-2c2b-4d33-9c2e-5e6e025b0979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:36.928987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c5c2f-f17b-4455-a8a5-89737eb6f750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:36.930486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c5c2f-f17b-4455-a8a5-89737eb6f750 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:36.936741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-89d65f32-2c2b-4d33-9c2e-5e6e025b0979 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:36.938309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89d65f32-2c2b-4d33-9c2e-5e6e025b0979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:36.939653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:37.048270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 383/746] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:36 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 22 11:01:37.050098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a0750-7222-4295-a8aa-7fffcc0d1790 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:37.050616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a0750-7222-4295-a8aa-7fffcc0d1790 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:37.051099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a0750-7222-4295-a8aa-7fffcc0d1790 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:37.052907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 384/747] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:37.058436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 364/748] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:36 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 22 11:01:37.064680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a0750-7222-4295-a8aa-7fffcc0d1790 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:37.065302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c6379-11a2-4754-9dbb-a21aff065be8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:37.065302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c6379-11a2-4754-9dbb-a21aff065be8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:37.065572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c6379-11a2-4754-9dbb-a21aff065be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:37.096549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-380c6379-11a2-4754-9dbb-a21aff065be8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:37.098500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-380c6379-11a2-4754-9dbb-a21aff065be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:37.174688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a0750-7222-4295-a8aa-7fffcc0d1790 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:37.199726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 365/749] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:37 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 22 11:01:37.199913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:37.201360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c6379-11a2-4754-9dbb-a21aff065be8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:37.230416 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b206c17-8d32-44c2-83af-7b8bdffce0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:37.232139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b206c17-8d32-44c2-83af-7b8bdffce0cc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:37.232897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b206c17-8d32-44c2-83af-7b8bdffce0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:37.233387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b206c17-8d32-44c2-83af-7b8bdffce0cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:37.246687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:37.251067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b206c17-8d32-44c2-83af-7b8bdffce0cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:37.253437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b206c17-8d32-44c2-83af-7b8bdffce0cc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-2b206c17-8d32-44c2-83af-7b8bdffce0cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:37.279535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:37.280475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:37.281373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:37.281373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:37.281373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:37.281373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:37.288106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 385/750] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:37.288106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b206c17-8d32-44c2-83af-7b8bdffce0cc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:37.338655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c6379-11a2-4754-9dbb-a21aff065be8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:37.392145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b206c17-8d32-44c2-83af-7b8bdffce0cc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:37.395336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b8944-ed16-457d-ad64-e364d088b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:37.396354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b8944-ed16-457d-ad64-e364d088b048 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:37.396855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b8944-ed16-457d-ad64-e364d088b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:37.397319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b8944-ed16-457d-ad64-e364d088b048 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:37.412091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b8944-ed16-457d-ad64-e364d088b048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-935b8944-ed16-457d-ad64-e364d088b048 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:37.423827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:37.424840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:37.425695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:37.428259 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 366/751] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:37.429280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b8944-ed16-457d-ad64-e364d088b048 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:37.435509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a6913-cfb3-49a7-9f32-27fb6737091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:37.436388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6913-cfb3-49a7-9f32-27fb6737091a None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:37.436781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6913-cfb3-49a7-9f32-27fb6737091a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:37.437308 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6913-cfb3-49a7-9f32-27fb6737091a None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:37.438147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a6913-cfb3-49a7-9f32-27fb6737091a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:37.439716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a6913-cfb3-49a7-9f32-27fb6737091a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:37.458823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 386/752] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:01:37 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:37.463438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a6913-cfb3-49a7-9f32-27fb6737091a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:37.511095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:37.632852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:37.642077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b8944-ed16-457d-ad64-e364d088b048 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:37.691818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a6913-cfb3-49a7-9f32-27fb6737091a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:37.707349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674106d-e9d6-4ffc-83a7-aa7a1980ceca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:37.708222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674106d-e9d6-4ffc-83a7-aa7a1980ceca None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:37.708584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674106d-e9d6-4ffc-83a7-aa7a1980ceca None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:37.708937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674106d-e9d6-4ffc-83a7-aa7a1980ceca None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:37.710011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674106d-e9d6-4ffc-83a7-aa7a1980ceca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:37.710411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674106d-e9d6-4ffc-83a7-aa7a1980ceca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:37.712721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674106d-e9d6-4ffc-83a7-aa7a1980ceca None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:37.731834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 367/753] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:37.737711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674106d-e9d6-4ffc-83a7-aa7a1980ceca None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:37.744849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc194b-0870-4327-b4e4-9598daa60072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:37.745806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc194b-0870-4327-b4e4-9598daa60072 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:37.746301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc194b-0870-4327-b4e4-9598daa60072 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:37.746761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc194b-0870-4327-b4e4-9598daa60072 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:37.747792 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc194b-0870-4327-b4e4-9598daa60072 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:37.748292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc194b-0870-4327-b4e4-9598daa60072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:37.750338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc194b-0870-4327-b4e4-9598daa60072 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:37.763204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 387/754] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:37.769827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc194b-0870-4327-b4e4-9598daa60072 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:37.872832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:37.879226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674106d-e9d6-4ffc-83a7-aa7a1980ceca None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:37.901084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc194b-0870-4327-b4e4-9598daa60072 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:37.922302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455debe-6afe-4d9d-b012-e7f6b6ddf60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:37.923491 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455debe-6afe-4d9d-b012-e7f6b6ddf60d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:37.924385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455debe-6afe-4d9d-b012-e7f6b6ddf60d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:37.924849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455debe-6afe-4d9d-b012-e7f6b6ddf60d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:37.934711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455debe-6afe-4d9d-b012-e7f6b6ddf60d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:37.937017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455debe-6afe-4d9d-b012-e7f6b6ddf60d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:37.940310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16555b-e072-43ae-8eb7-5e45d1e2cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:37.942677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16555b-e072-43ae-8eb7-5e45d1e2cac5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:37.942977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16555b-e072-43ae-8eb7-5e45d1e2cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:37.943355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16555b-e072-43ae-8eb7-5e45d1e2cac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3455debe-6afe-4d9d-b012-e7f6b6ddf60d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:37.954230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:37.955136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:37.956079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:37.956079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:37.956079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:37.956079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:37.956079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:37.956079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:37.956079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:37.956079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:37.957049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 368/755] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:37.963361 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16555b-e072-43ae-8eb7-5e45d1e2cac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:37.964000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16555b-e072-43ae-8eb7-5e45d1e2cac5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:37.969255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455debe-6afe-4d9d-b012-e7f6b6ddf60d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ae16555b-e072-43ae-8eb7-5e45d1e2cac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:37.996663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:37.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:37.998627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:37.998627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:37.998627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:37.998627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:38.000831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 388/756] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:38.005566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:38.010552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16555b-e072-43ae-8eb7-5e45d1e2cac5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:38.132141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455debe-6afe-4d9d-b012-e7f6b6ddf60d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:38.177865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d76af-40f7-4cc9-bbb9-45ffe82af22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:38.178894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d76af-40f7-4cc9-bbb9-45ffe82af22d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:38.179415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d76af-40f7-4cc9-bbb9-45ffe82af22d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:38.179901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d76af-40f7-4cc9-bbb9-45ffe82af22d None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:38.180883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d76af-40f7-4cc9-bbb9-45ffe82af22d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:38.181434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d76af-40f7-4cc9-bbb9-45ffe82af22d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:38.183074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d76af-40f7-4cc9-bbb9-45ffe82af22d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:38.199052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16555b-e072-43ae-8eb7-5e45d1e2cac5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:38.205504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 369/757] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:38.211457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d76af-40f7-4cc9-bbb9-45ffe82af22d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:38.228603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:38.239157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae939af-fefb-4e73-ab65-f4c4c68979f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:38.240228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae939af-fefb-4e73-ab65-f4c4c68979f7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:38.240720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae939af-fefb-4e73-ab65-f4c4c68979f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:38.241200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae939af-fefb-4e73-ab65-f4c4c68979f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:38.242171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae939af-fefb-4e73-ab65-f4c4c68979f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:38.242694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae939af-fefb-4e73-ab65-f4c4c68979f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:38.244315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae939af-fefb-4e73-ab65-f4c4c68979f7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:38.273767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 389/758] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:38.280070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae939af-fefb-4e73-ab65-f4c4c68979f7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:38.317248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:38.343370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d76af-40f7-4cc9-bbb9-45ffe82af22d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:38.380924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9a7ab-0f23-412d-a483-9146e9c2e6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:38.381599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9a7ab-0f23-412d-a483-9146e9c2e6fa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:38.382137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9a7ab-0f23-412d-a483-9146e9c2e6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:38.382425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9a7ab-0f23-412d-a483-9146e9c2e6fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:38.383289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9a7ab-0f23-412d-a483-9146e9c2e6fa None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=89224d740c7148938c2d803804083a57, user.id=e7935e85cc3f4ed1ad5b01ba892e6e82) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:38.383626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9a7ab-0f23-412d-a483-9146e9c2e6fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:38.385612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9a7ab-0f23-412d-a483-9146e9c2e6fa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:38.401744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 370/759] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:38 2022] GET /identity/v3/role_assignments?user.id=e7935e85cc3f4ed1ad5b01ba892e6e82&scope.project.id=89224d740c7148938c2d803804083a57&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:38.497747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae939af-fefb-4e73-ab65-f4c4c68979f7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:38.541714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518740f4-c331-46fd-bba4-bcdba176b50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:38.542543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518740f4-c331-46fd-bba4-bcdba176b50b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:38.542833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518740f4-c331-46fd-bba4-bcdba176b50b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:38.543109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518740f4-c331-46fd-bba4-bcdba176b50b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:38.551741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518740f4-c331-46fd-bba4-bcdba176b50b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:38.553232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518740f4-c331-46fd-bba4-bcdba176b50b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-518740f4-c331-46fd-bba4-bcdba176b50b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:38.579846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:38.580942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:38.581909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 390/760] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 22 11:01:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:38.587013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9a7ab-0f23-412d-a483-9146e9c2e6fa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:38.728911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9a7ab-0f23-412d-a483-9146e9c2e6fa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:38.771771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070153-7b65-4783-b135-f86ad5064011 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:38.772074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070153-7b65-4783-b135-f86ad5064011 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:38.772263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070153-7b65-4783-b135-f86ad5064011 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:38.773879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 391/761] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:38.780466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f932832-3d97-46ee-b216-bd97613fc392 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:38.780760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f932832-3d97-46ee-b216-bd97613fc392 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:38.781040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f932832-3d97-46ee-b216-bd97613fc392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:38.785308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b15657-785f-4202-a0eb-87ca2ea3b539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:38.785983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15657-785f-4202-a0eb-87ca2ea3b539 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:38.786270 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15657-785f-4202-a0eb-87ca2ea3b539 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:38.786550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15657-785f-4202-a0eb-87ca2ea3b539 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:38.787429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b15657-785f-4202-a0eb-87ca2ea3b539 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:38.787745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b15657-785f-4202-a0eb-87ca2ea3b539 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:38.789188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b15657-785f-4202-a0eb-87ca2ea3b539 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:38.806955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:38.820991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4f932832-3d97-46ee-b216-bd97613fc392 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:38.822841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4f932832-3d97-46ee-b216-bd97613fc392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:38.825101 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 371/762] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 22 11:01:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:38.826824 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b15657-785f-4202-a0eb-87ca2ea3b539 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:38.912527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:38.962001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 392/763] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:38.991679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f932832-3d97-46ee-b216-bd97613fc392 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:39.004718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b15657-785f-4202-a0eb-87ca2ea3b539 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:39.088565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf0620-b756-44c1-a48a-be8f953aa571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:39.089384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf0620-b756-44c1-a48a-be8f953aa571 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.089676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf0620-b756-44c1-a48a-be8f953aa571 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.089958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf0620-b756-44c1-a48a-be8f953aa571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.120516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf0620-b756-44c1-a48a-be8f953aa571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:39.122453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f932832-3d97-46ee-b216-bd97613fc392 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:39.127516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf0620-b756-44c1-a48a-be8f953aa571 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c0bf0620-b756-44c1-a48a-be8f953aa571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:39.157638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:39.158577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:39.159528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:39.159528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:39.159528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:39.159528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:39.159528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 372/764] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:39.161307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8f706-64a9-43c7-b37a-51e2db432675 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.161307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8f706-64a9-43c7-b37a-51e2db432675 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.161307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8f706-64a9-43c7-b37a-51e2db432675 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.166803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 373/765] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:39.167607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8f706-64a9-43c7-b37a-51e2db432675 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:39.175128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521d5b2-4272-4f1c-bf60-9db232e7f433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:39.175128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521d5b2-4272-4f1c-bf60-9db232e7f433 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.175128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521d5b2-4272-4f1c-bf60-9db232e7f433 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.175128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521d5b2-4272-4f1c-bf60-9db232e7f433 None admin] PATH_INFO: `/v3/services/network` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.175520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521d5b2-4272-4f1c-bf60-9db232e7f433 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:39.176035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521d5b2-4272-4f1c-bf60-9db232e7f433 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e521d5b2-4272-4f1c-bf60-9db232e7f433 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:39.191503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:39.192509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 22 11:01:39.193380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:39.196407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 393/766] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:38 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:39.202404 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e62fe-bc38-4de2-8d04-1b51d8f6c75e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.203426 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e62fe-bc38-4de2-8d04-1b51d8f6c75e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.203794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e62fe-bc38-4de2-8d04-1b51d8f6c75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.228988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c69e62fe-bc38-4de2-8d04-1b51d8f6c75e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:39.229195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:39.230806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c69e62fe-bc38-4de2-8d04-1b51d8f6c75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:39.280178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:39.289392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8f706-64a9-43c7-b37a-51e2db432675 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:39.327548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c89e2-07a8-4394-bb18-059f56f1d1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:39.328966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c89e2-07a8-4394-bb18-059f56f1d1db None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.329456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c89e2-07a8-4394-bb18-059f56f1d1db None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.329962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c89e2-07a8-4394-bb18-059f56f1d1db None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.331004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c89e2-07a8-4394-bb18-059f56f1d1db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:39.331359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 394/767] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:39.331804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c89e2-07a8-4394-bb18-059f56f1d1db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:39.332241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e62fe-bc38-4de2-8d04-1b51d8f6c75e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:39.333685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c89e2-07a8-4394-bb18-059f56f1d1db None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:39.360804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 374/768] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:39.366750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c89e2-07a8-4394-bb18-059f56f1d1db None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:39.458464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e62fe-bc38-4de2-8d04-1b51d8f6c75e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:39.487870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c89e2-07a8-4394-bb18-059f56f1d1db None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:39.496914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609af66d-5ad2-4a88-b2b9-2b38dfc2362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:39.497892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609af66d-5ad2-4a88-b2b9-2b38dfc2362a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.498816 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609af66d-5ad2-4a88-b2b9-2b38dfc2362a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.499347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609af66d-5ad2-4a88-b2b9-2b38dfc2362a None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.500142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:39.500515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609af66d-5ad2-4a88-b2b9-2b38dfc2362a None admin] RBAC: Adding query filter params (name=network) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:39.501023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609af66d-5ad2-4a88-b2b9-2b38dfc2362a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:39.502585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609af66d-5ad2-4a88-b2b9-2b38dfc2362a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:39.509144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 395/769] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:01:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:39.510225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609af66d-5ad2-4a88-b2b9-2b38dfc2362a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:39.537402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a78ec5-65c1-498f-acb5-8ef353d23f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:39.538212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a78ec5-65c1-498f-acb5-8ef353d23f94 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.538503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a78ec5-65c1-498f-acb5-8ef353d23f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.538784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a78ec5-65c1-498f-acb5-8ef353d23f94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.548192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a78ec5-65c1-498f-acb5-8ef353d23f94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:39.552016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a78ec5-65c1-498f-acb5-8ef353d23f94 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:39.563516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b6a78ec5-65c1-498f-acb5-8ef353d23f94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:39.572840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:39.573777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:39.574686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:39.574686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:39.574686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:39.574686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:39.574686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:39.574686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:39.574686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:39.574686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:39.579343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 375/770] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:39.580530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a78ec5-65c1-498f-acb5-8ef353d23f94 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:39.650545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609af66d-5ad2-4a88-b2b9-2b38dfc2362a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:39.695417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef2c92-742a-43af-b165-ed221eec169f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:39.697295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef2c92-742a-43af-b165-ed221eec169f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.697742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef2c92-742a-43af-b165-ed221eec169f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.698176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef2c92-742a-43af-b165-ed221eec169f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.713624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef2c92-742a-43af-b165-ed221eec169f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:39.713987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a78ec5-65c1-498f-acb5-8ef353d23f94 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4fef2c92-742a-43af-b165-ed221eec169f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:39.723622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:39.724806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:39.725656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:39.729042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 396/771] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:39.730104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef2c92-742a-43af-b165-ed221eec169f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:39.761557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03836328-979f-4131-8307-90cd9e5859b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:39.762640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03836328-979f-4131-8307-90cd9e5859b4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.763087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03836328-979f-4131-8307-90cd9e5859b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.763853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03836328-979f-4131-8307-90cd9e5859b4 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.765002 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03836328-979f-4131-8307-90cd9e5859b4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:39.765501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03836328-979f-4131-8307-90cd9e5859b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:39.767810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03836328-979f-4131-8307-90cd9e5859b4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:39.780031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 376/772] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:01:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:39.781115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03836328-979f-4131-8307-90cd9e5859b4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:39.821481 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:39.857801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:39.865984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef2c92-742a-43af-b165-ed221eec169f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:39.911916 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03836328-979f-4131-8307-90cd9e5859b4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:39.912369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdf489-2df3-49e2-976e-32bff93f92f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:39.913204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdf489-2df3-49e2-976e-32bff93f92f6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.913904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdf489-2df3-49e2-976e-32bff93f92f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.914356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdf489-2df3-49e2-976e-32bff93f92f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.916631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdf489-2df3-49e2-976e-32bff93f92f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:39.916950 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdf489-2df3-49e2-976e-32bff93f92f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:39.919590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdf489-2df3-49e2-976e-32bff93f92f6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:39.943411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 397/773] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:39.944994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdf489-2df3-49e2-976e-32bff93f92f6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:39.960446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23099d-23a1-49a4-8072-949b4f29d233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:39.961178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23099d-23a1-49a4-8072-949b4f29d233 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:39.961504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23099d-23a1-49a4-8072-949b4f29d233 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:39.962026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23099d-23a1-49a4-8072-949b4f29d233 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:39.962829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23099d-23a1-49a4-8072-949b4f29d233 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:39.963142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23099d-23a1-49a4-8072-949b4f29d233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:39.964838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23099d-23a1-49a4-8072-949b4f29d233 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:39.975460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 377/774] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:39.979846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23099d-23a1-49a4-8072-949b4f29d233 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:40.062839 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdf489-2df3-49e2-976e-32bff93f92f6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:40.086884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:40.094221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23099d-23a1-49a4-8072-949b4f29d233 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:40.099634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf857e1f-3e57-4237-aab3-45ddeed74bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:40.100305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf857e1f-3e57-4237-aab3-45ddeed74bcd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.100596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf857e1f-3e57-4237-aab3-45ddeed74bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.100880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf857e1f-3e57-4237-aab3-45ddeed74bcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.101680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf857e1f-3e57-4237-aab3-45ddeed74bcd None admin] RBAC: Adding query filter params (interface=public, service_id=6ee3efd365124a6a9f6f74295c8e7ce8, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:40.101997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf857e1f-3e57-4237-aab3-45ddeed74bcd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:40.103930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf857e1f-3e57-4237-aab3-45ddeed74bcd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:40.111667 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 398/775] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 22 11:01:39 2022] GET /identity/v3/endpoints?service_id=6ee3efd365124a6a9f6f74295c8e7ce8&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:40.112471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf857e1f-3e57-4237-aab3-45ddeed74bcd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:40.134637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591e9f4-4c13-4f88-b882-25427b9be807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:40.135943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591e9f4-4c13-4f88-b882-25427b9be807 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.136390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591e9f4-4c13-4f88-b882-25427b9be807 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.136860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591e9f4-4c13-4f88-b882-25427b9be807 None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/c51929d0ecb047b4bdc405dc86fe7eb6/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.137144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:40.139420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591e9f4-4c13-4f88-b882-25427b9be807 None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=c51929d0ecb047b4bdc405dc86fe7eb6, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:40.141097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591e9f4-4c13-4f88-b882-25427b9be807 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:40.159292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 378/776] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:01:39 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/c51929d0ecb047b4bdc405dc86fe7eb6/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:01:40.159292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5ba25-87a5-467c-8c52-0ca2a3a57235 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.159292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5ba25-87a5-467c-8c52-0ca2a3a57235 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.159292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5ba25-87a5-467c-8c52-0ca2a3a57235 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.159292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 379/777] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:40.159292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5ba25-87a5-467c-8c52-0ca2a3a57235 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:40.238561 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf857e1f-3e57-4237-aab3-45ddeed74bcd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:40.278493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88072c-feee-4e22-afe4-60e8ddc0be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:40.279338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88072c-feee-4e22-afe4-60e8ddc0be75 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.279634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88072c-feee-4e22-afe4-60e8ddc0be75 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.279974 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88072c-feee-4e22-afe4-60e8ddc0be75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.294855 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88072c-feee-4e22-afe4-60e8ddc0be75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:40.295967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5ba25-87a5-467c-8c52-0ca2a3a57235 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:40.296436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88072c-feee-4e22-afe4-60e8ddc0be75 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0f88072c-feee-4e22-afe4-60e8ddc0be75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:40.316543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:40.317517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:40.318499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:40.318499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:40.318499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:40.318499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:40.318499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 399/778] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:40.319211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eaca3-a7d9-40bd-89a3-04d2c6af883f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.319403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eaca3-a7d9-40bd-89a3-04d2c6af883f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.322237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eaca3-a7d9-40bd-89a3-04d2c6af883f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.340501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-715eaca3-a7d9-40bd-89a3-04d2c6af883f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:40.341731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-715eaca3-a7d9-40bd-89a3-04d2c6af883f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:40.345183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5998b58-20cd-48e2-85d4-82c27a81a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:40.346309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5998b58-20cd-48e2-85d4-82c27a81a60f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.347548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5998b58-20cd-48e2-85d4-82c27a81a60f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.347548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5998b58-20cd-48e2-85d4-82c27a81a60f None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.347742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5998b58-20cd-48e2-85d4-82c27a81a60f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:40.349732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5998b58-20cd-48e2-85d4-82c27a81a60f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:40.357984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 380/779] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 22 11:01:40 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:40.359027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5998b58-20cd-48e2-85d4-82c27a81a60f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:40.391836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:40.441774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 400/780] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:40.466157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715eaca3-a7d9-40bd-89a3-04d2c6af883f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:40.468656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:40.521510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5998b58-20cd-48e2-85d4-82c27a81a60f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:40.565503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d4c4c-c475-48eb-b292-3e7177051317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:40.566498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d4c4c-c475-48eb-b292-3e7177051317 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.566928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d4c4c-c475-48eb-b292-3e7177051317 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.567417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d4c4c-c475-48eb-b292-3e7177051317 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.568813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d4c4c-c475-48eb-b292-3e7177051317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:40.569271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d4c4c-c475-48eb-b292-3e7177051317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:40.571251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d4c4c-c475-48eb-b292-3e7177051317 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:40.590027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 381/781] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:40.603421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d4c4c-c475-48eb-b292-3e7177051317 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:40.641998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715eaca3-a7d9-40bd-89a3-04d2c6af883f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:40.692965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea5743-3127-4ec6-b87f-6ca4757de6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:40.693935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea5743-3127-4ec6-b87f-6ca4757de6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.694221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea5743-3127-4ec6-b87f-6ca4757de6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.694492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea5743-3127-4ec6-b87f-6ca4757de6a2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.695153 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea5743-3127-4ec6-b87f-6ca4757de6a2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:40.697064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea5743-3127-4ec6-b87f-6ca4757de6a2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-36ea5743-3127-4ec6-b87f-6ca4757de6a2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:40.713312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:40.714360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 22 11:01:40.715418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:40.717034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:40.717186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 401/782] 10.222.0.20 () {56 vars in 1129 bytes} [Fri Jul 22 11:01:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:40.720535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770d85a-f604-432b-9ece-a8b84f92d286 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.720822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770d85a-f604-432b-9ece-a8b84f92d286 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.721114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770d85a-f604-432b-9ece-a8b84f92d286 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.722641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 402/783] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:40.727294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770d85a-f604-432b-9ece-a8b84f92d286 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:40.742220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d4c4c-c475-48eb-b292-3e7177051317 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:40.781123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1c269-411f-45c8-bf0e-04273def1985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:40.782280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1c269-411f-45c8-bf0e-04273def1985 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.782787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1c269-411f-45c8-bf0e-04273def1985 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.783304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1c269-411f-45c8-bf0e-04273def1985 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.791277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1c269-411f-45c8-bf0e-04273def1985 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:40.793139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1c269-411f-45c8-bf0e-04273def1985 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:40.796107 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-92c1c269-411f-45c8-bf0e-04273def1985 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:40.810020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:40.811076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:40.812065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:40.814714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 382/784] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:01:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:40.816206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6ae71-6e46-4e66-bf3a-04ace5abf498 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.816706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6ae71-6e46-4e66-bf3a-04ace5abf498 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.817205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6ae71-6e46-4e66-bf3a-04ace5abf498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.834785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770d85a-f604-432b-9ece-a8b84f92d286 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:40.835196 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-59b6ae71-6e46-4e66-bf3a-04ace5abf498 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:40.836649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-59b6ae71-6e46-4e66-bf3a-04ace5abf498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:40.871651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af182a-9ecb-4c40-af56-11382e13e9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:40.872320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af182a-9ecb-4c40-af56-11382e13e9bd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:40.872610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af182a-9ecb-4c40-af56-11382e13e9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:40.872888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af182a-9ecb-4c40-af56-11382e13e9bd None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:40.873587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af182a-9ecb-4c40-af56-11382e13e9bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:40.873893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af182a-9ecb-4c40-af56-11382e13e9bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:40.875250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af182a-9ecb-4c40-af56-11382e13e9bd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:40.885206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 403/785] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 22 11:01:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:40.886116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af182a-9ecb-4c40-af56-11382e13e9bd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:40.912202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 383/786] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:40.920332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6ae71-6e46-4e66-bf3a-04ace5abf498 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:40.999374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af182a-9ecb-4c40-af56-11382e13e9bd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.017671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:41.034773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:41.042527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8d673-50d2-4618-af11-eaba506d3419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.043512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8d673-50d2-4618-af11-eaba506d3419 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.044060 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8d673-50d2-4618-af11-eaba506d3419 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.044344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8d673-50d2-4618-af11-eaba506d3419 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.045200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8d673-50d2-4618-af11-eaba506d3419 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:41.045515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8d673-50d2-4618-af11-eaba506d3419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:41.046957 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8d673-50d2-4618-af11-eaba506d3419 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:41.060599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6ae71-6e46-4e66-bf3a-04ace5abf498 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.081906 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 404/787] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 22 11:01:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:41.081906 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8d673-50d2-4618-af11-eaba506d3419 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:41.122281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6502145-aaaa-4b40-b4f7-0cc607fd201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.123837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6502145-aaaa-4b40-b4f7-0cc607fd201a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.125499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6502145-aaaa-4b40-b4f7-0cc607fd201a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.126017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6502145-aaaa-4b40-b4f7-0cc607fd201a None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.127322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6502145-aaaa-4b40-b4f7-0cc607fd201a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:41.128034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6502145-aaaa-4b40-b4f7-0cc607fd201a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:41.130657 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6502145-aaaa-4b40-b4f7-0cc607fd201a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:41.140171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 384/788] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 22 11:01:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:41.141568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6502145-aaaa-4b40-b4f7-0cc607fd201a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:41.211009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8d673-50d2-4618-af11-eaba506d3419 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.256314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae1592-7743-4ace-9a4a-1e07c6e6a24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.257626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1592-7743-4ace-9a4a-1e07c6e6a24c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.258112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1592-7743-4ace-9a4a-1e07c6e6a24c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.258761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1592-7743-4ace-9a4a-1e07c6e6a24c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.264365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6502145-aaaa-4b40-b4f7-0cc607fd201a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.274521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae1592-7743-4ace-9a4a-1e07c6e6a24c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a4ae1592-7743-4ace-9a4a-1e07c6e6a24c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:41.285365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:41.286292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:41.287224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:41.291334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 405/789] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:41.292475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae1592-7743-4ace-9a4a-1e07c6e6a24c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:41.309851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e571e9b-8422-48fa-8204-9a504b441887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.311354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e571e9b-8422-48fa-8204-9a504b441887 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.312137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e571e9b-8422-48fa-8204-9a504b441887 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.312634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e571e9b-8422-48fa-8204-9a504b441887 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.315722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:41.327391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e571e9b-8422-48fa-8204-9a504b441887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:41.329889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e571e9b-8422-48fa-8204-9a504b441887 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6e571e9b-8422-48fa-8204-9a504b441887 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:41.354516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:41.355938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:41.356814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:41.356814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:41.356814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:41.356814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:41.356814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 385/790] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:41.357095 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e571e9b-8422-48fa-8204-9a504b441887 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:41.366700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:41.436849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae1592-7743-4ace-9a4a-1e07c6e6a24c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.480804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e442e4b-0bf5-40db-afbe-3cb0198dcb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.481470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442e4b-0bf5-40db-afbe-3cb0198dcb86 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.481768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442e4b-0bf5-40db-afbe-3cb0198dcb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.482437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442e4b-0bf5-40db-afbe-3cb0198dcb86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.483364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442e4b-0bf5-40db-afbe-3cb0198dcb86 None admin] RBAC: Adding query filter params (interface=public, service_id=0237d339915f4a689b72214eaad96289, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:41.483765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442e4b-0bf5-40db-afbe-3cb0198dcb86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:41.485467 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442e4b-0bf5-40db-afbe-3cb0198dcb86 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:41.495076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 406/791] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 22 11:01:41 2022] GET /identity/v3/endpoints?service_id=0237d339915f4a689b72214eaad96289&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:41.496352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e442e4b-0bf5-40db-afbe-3cb0198dcb86 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:41.519556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e571e9b-8422-48fa-8204-9a504b441887 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.562503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c498d-5925-46ef-9f1a-87cf678f26a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.563851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c498d-5925-46ef-9f1a-87cf678f26a2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.564580 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c498d-5925-46ef-9f1a-87cf678f26a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.565008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c498d-5925-46ef-9f1a-87cf678f26a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.566060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c498d-5925-46ef-9f1a-87cf678f26a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:41.566531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c498d-5925-46ef-9f1a-87cf678f26a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:41.568600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c498d-5925-46ef-9f1a-87cf678f26a2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:41.579792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 386/792] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:41.580879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c498d-5925-46ef-9f1a-87cf678f26a2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:41.630547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e442e4b-0bf5-40db-afbe-3cb0198dcb86 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.645135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:41.647392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:41.676206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34907ea9-f4d8-444b-82fa-e3a653125584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.676966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34907ea9-f4d8-444b-82fa-e3a653125584 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.677355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34907ea9-f4d8-444b-82fa-e3a653125584 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.677771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34907ea9-f4d8-444b-82fa-e3a653125584 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.679011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34907ea9-f4d8-444b-82fa-e3a653125584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:41.679457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34907ea9-f4d8-444b-82fa-e3a653125584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:41.681547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34907ea9-f4d8-444b-82fa-e3a653125584 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:41.700168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 407/793] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:41.701241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34907ea9-f4d8-444b-82fa-e3a653125584 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:41.709409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c498d-5925-46ef-9f1a-87cf678f26a2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.748638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80366fb3-7096-4dc3-8dfe-94aee75cb953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.749357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80366fb3-7096-4dc3-8dfe-94aee75cb953 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.749703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80366fb3-7096-4dc3-8dfe-94aee75cb953 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.750124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80366fb3-7096-4dc3-8dfe-94aee75cb953 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.750731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80366fb3-7096-4dc3-8dfe-94aee75cb953 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:41.752172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80366fb3-7096-4dc3-8dfe-94aee75cb953 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:41.758792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 387/794] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 22 11:01:41 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:41.759812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80366fb3-7096-4dc3-8dfe-94aee75cb953 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:41.828624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34907ea9-f4d8-444b-82fa-e3a653125584 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.892135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869e1c5-3996-40ce-badb-f5d90f0e9e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.893361 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869e1c5-3996-40ce-badb-f5d90f0e9e7b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.893881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869e1c5-3996-40ce-badb-f5d90f0e9e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.894378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869e1c5-3996-40ce-badb-f5d90f0e9e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.913789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80366fb3-7096-4dc3-8dfe-94aee75cb953 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:41.917089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869e1c5-3996-40ce-badb-f5d90f0e9e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:41.920495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869e1c5-3996-40ce-badb-f5d90f0e9e7b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:41.950334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4869e1c5-3996-40ce-badb-f5d90f0e9e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:41.956841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:41.958117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:41.959007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:41.959007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:41.959007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:41.959007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:41.967591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 408/795] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:41.973540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869e1c5-3996-40ce-badb-f5d90f0e9e7b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:41.979251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03482b-8a2d-4e65-adcd-455a562d4947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:41.980110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03482b-8a2d-4e65-adcd-455a562d4947 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:41.980399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03482b-8a2d-4e65-adcd-455a562d4947 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:41.981496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03482b-8a2d-4e65-adcd-455a562d4947 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:41.990158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03482b-8a2d-4e65-adcd-455a562d4947 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:41.993106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03482b-8a2d-4e65-adcd-455a562d4947 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:42.011930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0f03482b-8a2d-4e65-adcd-455a562d4947 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:42.014975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:42.016352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:42.017314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:42.017314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:42.017314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:42.017314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:42.017314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:42.017314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:42.017314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:42.017314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:42.017314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 388/796] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:42.022434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03482b-8a2d-4e65-adcd-455a562d4947 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:42.115284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869e1c5-3996-40ce-badb-f5d90f0e9e7b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:42.147249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03482b-8a2d-4e65-adcd-455a562d4947 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:42.153509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89477bac-49a7-44b6-a80e-364a308f65df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:42.154194 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89477bac-49a7-44b6-a80e-364a308f65df None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.154486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89477bac-49a7-44b6-a80e-364a308f65df None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.154770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89477bac-49a7-44b6-a80e-364a308f65df None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.155533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89477bac-49a7-44b6-a80e-364a308f65df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:42.155846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89477bac-49a7-44b6-a80e-364a308f65df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:42.157229 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89477bac-49a7-44b6-a80e-364a308f65df None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:42.174623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 409/797] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:42.184414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89477bac-49a7-44b6-a80e-364a308f65df None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:42.194305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b98791-da8a-4132-bebf-b0e8276c4832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:42.195634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b98791-da8a-4132-bebf-b0e8276c4832 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.196142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b98791-da8a-4132-bebf-b0e8276c4832 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.197525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b98791-da8a-4132-bebf-b0e8276c4832 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.198583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b98791-da8a-4132-bebf-b0e8276c4832 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:42.199094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b98791-da8a-4132-bebf-b0e8276c4832 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:42.200870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b98791-da8a-4132-bebf-b0e8276c4832 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:42.219017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 389/798] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:42.224506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b98791-da8a-4132-bebf-b0e8276c4832 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:42.258433 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:42.335068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b98791-da8a-4132-bebf-b0e8276c4832 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:42.363121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:42.373325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7712f99-8140-46b8-bed9-fb7487ba88b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:42.374840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712f99-8140-46b8-bed9-fb7487ba88b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.375312 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712f99-8140-46b8-bed9-fb7487ba88b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.375730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712f99-8140-46b8-bed9-fb7487ba88b1 None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/646d7c5e6c8e4825a7fe06d018ee319c/roles/a9580499eef24770867aeac1602aa002` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.393409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7712f99-8140-46b8-bed9-fb7487ba88b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=646d7c5e6c8e4825a7fe06d018ee319c, role_id=a9580499eef24770867aeac1602aa002)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:42.395164 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7712f99-8140-46b8-bed9-fb7487ba88b1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:42.395419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89477bac-49a7-44b6-a80e-364a308f65df None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:42.413969 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 390/799] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:01:42 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/646d7c5e6c8e4825a7fe06d018ee319c/roles/a9580499eef24770867aeac1602aa002 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:01:42.446139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602a843-56c4-404d-b68b-faa2d0e1fea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:42.447964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602a843-56c4-404d-b68b-faa2d0e1fea3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.448403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602a843-56c4-404d-b68b-faa2d0e1fea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.448918 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602a843-56c4-404d-b68b-faa2d0e1fea3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.457984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602a843-56c4-404d-b68b-faa2d0e1fea3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:42.459864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602a843-56c4-404d-b68b-faa2d0e1fea3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5602a843-56c4-404d-b68b-faa2d0e1fea3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:42.483342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:42.484555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:42.485713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:42.488864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16a4e9-794d-4997-ab0e-b7d685de3956 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.489151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 410/800] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:42.489727 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16a4e9-794d-4997-ab0e-b7d685de3956 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.490331 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16a4e9-794d-4997-ab0e-b7d685de3956 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.494600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 391/801] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:01:42.499444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602a843-56c4-404d-b68b-faa2d0e1fea3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:42.502920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f801f3e-61b0-4935-b05e-cfc3b8960ef9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.503613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f801f3e-61b0-4935-b05e-cfc3b8960ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.504121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f801f3e-61b0-4935-b05e-cfc3b8960ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.524962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7f801f3e-61b0-4935-b05e-cfc3b8960ef9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:42.526757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7f801f3e-61b0-4935-b05e-cfc3b8960ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:42.645429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 392/802] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:01:42.654486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602a843-56c4-404d-b68b-faa2d0e1fea3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:42.671287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f801f3e-61b0-4935-b05e-cfc3b8960ef9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:42.689202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:42.701688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a5409-f1ee-4fcd-b785-20188972855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:42.702658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a5409-f1ee-4fcd-b785-20188972855f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.703143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a5409-f1ee-4fcd-b785-20188972855f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.703689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a5409-f1ee-4fcd-b785-20188972855f None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.704958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a5409-f1ee-4fcd-b785-20188972855f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:42.705483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a5409-f1ee-4fcd-b785-20188972855f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:42.707234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a5409-f1ee-4fcd-b785-20188972855f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:42.726493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 411/803] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 22 11:01:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:42.731712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a5409-f1ee-4fcd-b785-20188972855f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:42.734738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:42.784219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f801f3e-61b0-4935-b05e-cfc3b8960ef9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:42.821493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e5123-575a-47aa-a064-737fc529f61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:42.822645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e5123-575a-47aa-a064-737fc529f61b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.823131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e5123-575a-47aa-a064-737fc529f61b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.823630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e5123-575a-47aa-a064-737fc529f61b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.834768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e5123-575a-47aa-a064-737fc529f61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f67e5123-575a-47aa-a064-737fc529f61b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:42.843158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:42.844151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:42.844999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:42.847791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 393/804] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:42.855285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e5123-575a-47aa-a064-737fc529f61b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:42.896671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a5409-f1ee-4fcd-b785-20188972855f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:42.933708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121508bd-c941-43e9-89c1-043936b688e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:42.934279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121508bd-c941-43e9-89c1-043936b688e9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.934569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121508bd-c941-43e9-89c1-043936b688e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.934847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121508bd-c941-43e9-89c1-043936b688e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.938498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:42.947842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121508bd-c941-43e9-89c1-043936b688e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:42.949635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121508bd-c941-43e9-89c1-043936b688e9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-121508bd-c941-43e9-89c1-043936b688e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:42.969216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:42.970253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:42.971142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:42.971142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:42.971142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:42.971142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:42.971418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e5123-575a-47aa-a064-737fc529f61b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:42.973517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 412/805] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:42.975094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206d9b1-548e-4eb7-a175-34b63654e37c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:42.975605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206d9b1-548e-4eb7-a175-34b63654e37c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:42.976067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206d9b1-548e-4eb7-a175-34b63654e37c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:42.978317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 413/806] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:42.979389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206d9b1-548e-4eb7-a175-34b63654e37c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.011731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41031409-31a5-49bb-a0ab-930c2ba2a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.012534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41031409-31a5-49bb-a0ab-930c2ba2a699 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.012889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41031409-31a5-49bb-a0ab-930c2ba2a699 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.013252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41031409-31a5-49bb-a0ab-930c2ba2a699 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.014168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41031409-31a5-49bb-a0ab-930c2ba2a699 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:43.014562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41031409-31a5-49bb-a0ab-930c2ba2a699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.016094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41031409-31a5-49bb-a0ab-930c2ba2a699 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.025297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 394/807] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:43.026917 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3f4b2-3bf2-455a-8675-9a3a1e155157 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.027285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3f4b2-3bf2-455a-8675-9a3a1e155157 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.027685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3f4b2-3bf2-455a-8675-9a3a1e155157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.044174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:43.046960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ecb3f4b2-3bf2-455a-8675-9a3a1e155157 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:43.048406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ecb3f4b2-3bf2-455a-8675-9a3a1e155157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:43.094208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206d9b1-548e-4eb7-a175-34b63654e37c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:43.125913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 395/808] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:43.126913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3f4b2-3bf2-455a-8675-9a3a1e155157 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.134241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641f9ff-f819-465a-94ef-88f497e0bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.135669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641f9ff-f819-465a-94ef-88f497e0bd6f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.136205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641f9ff-f819-465a-94ef-88f497e0bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.136804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641f9ff-f819-465a-94ef-88f497e0bd6f None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.137826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641f9ff-f819-465a-94ef-88f497e0bd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:43.138952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641f9ff-f819-465a-94ef-88f497e0bd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.139917 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641f9ff-f819-465a-94ef-88f497e0bd6f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.153160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 414/809] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:43.158054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641f9ff-f819-465a-94ef-88f497e0bd6f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.198981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:43.239906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3f4b2-3bf2-455a-8675-9a3a1e155157 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:43.277934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75ab9-cc1c-4681-aeef-80662932ae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.278729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75ab9-cc1c-4681-aeef-80662932ae20 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.279011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75ab9-cc1c-4681-aeef-80662932ae20 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.279310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75ab9-cc1c-4681-aeef-80662932ae20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.283639 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641f9ff-f819-465a-94ef-88f497e0bd6f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:43.295222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b75ab9-cc1c-4681-aeef-80662932ae20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.297418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b75ab9-cc1c-4681-aeef-80662932ae20 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.300553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a2b75ab9-cc1c-4681-aeef-80662932ae20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:43.325266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:43.326486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:43.327564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:43.327564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:43.327564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:43.327564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:43.333358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 396/810] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:43.333358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75ab9-cc1c-4681-aeef-80662932ae20 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.334043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde1e44-3d10-4fd7-a5c0-7a182e45c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.334840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde1e44-3d10-4fd7-a5c0-7a182e45c541 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.335126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde1e44-3d10-4fd7-a5c0-7a182e45c541 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.335446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde1e44-3d10-4fd7-a5c0-7a182e45c541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.345069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde1e44-3d10-4fd7-a5c0-7a182e45c541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.346517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde1e44-3d10-4fd7-a5c0-7a182e45c541 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-efde1e44-3d10-4fd7-a5c0-7a182e45c541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:43.366464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:43.367500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:43.368446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:43.368446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:43.368446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:43.368446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:43.368446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:43.368446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:43.368446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:43.368446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:43.368446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 415/811] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:43.369510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde1e44-3d10-4fd7-a5c0-7a182e45c541 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.473210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75ab9-cc1c-4681-aeef-80662932ae20 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:43.493581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde1e44-3d10-4fd7-a5c0-7a182e45c541 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:43.496751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:43.519180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d6fa6-7e19-44cb-964c-c79d298bf1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.520899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d6fa6-7e19-44cb-964c-c79d298bf1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.521646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d6fa6-7e19-44cb-964c-c79d298bf1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.522151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d6fa6-7e19-44cb-964c-c79d298bf1e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.523252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d6fa6-7e19-44cb-964c-c79d298bf1e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.525946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d6fa6-7e19-44cb-964c-c79d298bf1e5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.532344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae80912-1e11-4975-856f-627d0deac52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.533093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae80912-1e11-4975-856f-627d0deac52d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.533457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae80912-1e11-4975-856f-627d0deac52d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.533811 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae80912-1e11-4975-856f-627d0deac52d None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.534608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae80912-1e11-4975-856f-627d0deac52d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:43.534986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae80912-1e11-4975-856f-627d0deac52d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.536477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae80912-1e11-4975-856f-627d0deac52d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-cb2d6fa6-7e19-44cb-964c-c79d298bf1e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:43.541495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:43.542442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 22 11:01:43.543287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:43.546738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 397/812] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:43 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:43.548049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d6fa6-7e19-44cb-964c-c79d298bf1e5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.554176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 416/813] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:43.555145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae80912-1e11-4975-856f-627d0deac52d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.598718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:43.673793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae80912-1e11-4975-856f-627d0deac52d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:43.691862 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d6fa6-7e19-44cb-964c-c79d298bf1e5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:43.717988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da2e21-7c8e-456e-9e50-ad013594c8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.719314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da2e21-7c8e-456e-9e50-ad013594c8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.720237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da2e21-7c8e-456e-9e50-ad013594c8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.721105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da2e21-7c8e-456e-9e50-ad013594c8e1 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.722377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da2e21-7c8e-456e-9e50-ad013594c8e1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:43.723409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da2e21-7c8e-456e-9e50-ad013594c8e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.725585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da2e21-7c8e-456e-9e50-ad013594c8e1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.735164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 417/814] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:01:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:43.735164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da2e21-7c8e-456e-9e50-ad013594c8e1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.739183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a73713-3a2c-458a-b2c7-5d9ad073108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.740314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a73713-3a2c-458a-b2c7-5d9ad073108c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.740314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a73713-3a2c-458a-b2c7-5d9ad073108c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.740508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a73713-3a2c-458a-b2c7-5d9ad073108c None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.741230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a73713-3a2c-458a-b2c7-5d9ad073108c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:43.741540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a73713-3a2c-458a-b2c7-5d9ad073108c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.742973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a73713-3a2c-458a-b2c7-5d9ad073108c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.756928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 398/815] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:43.757851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a73713-3a2c-458a-b2c7-5d9ad073108c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.810605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:43.842700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:43.846702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da2e21-7c8e-456e-9e50-ad013594c8e1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:43.885773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391f0a1-2b76-4223-af70-47bc8e9413ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.886570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391f0a1-2b76-4223-af70-47bc8e9413ac None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.886857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391f0a1-2b76-4223-af70-47bc8e9413ac None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.887135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391f0a1-2b76-4223-af70-47bc8e9413ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.892566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a73713-3a2c-458a-b2c7-5d9ad073108c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:43.894932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391f0a1-2b76-4223-af70-47bc8e9413ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.896403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391f0a1-2b76-4223-af70-47bc8e9413ac None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0391f0a1-2b76-4223-af70-47bc8e9413ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:43.911344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:43.912355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 418/816] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 22 11:01:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:43.913611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391f0a1-2b76-4223-af70-47bc8e9413ac None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:43.934633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd794bb-82bc-4b64-9ce6-5b39ebb138a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:43.935436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd794bb-82bc-4b64-9ce6-5b39ebb138a4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:43.935795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd794bb-82bc-4b64-9ce6-5b39ebb138a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:43.936148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd794bb-82bc-4b64-9ce6-5b39ebb138a4 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:43.937022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd794bb-82bc-4b64-9ce6-5b39ebb138a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:43.937401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd794bb-82bc-4b64-9ce6-5b39ebb138a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:43.938986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd794bb-82bc-4b64-9ce6-5b39ebb138a4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:43.944979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 399/817] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:01:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:43.946089 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd794bb-82bc-4b64-9ce6-5b39ebb138a4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:44.021030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391f0a1-2b76-4223-af70-47bc8e9413ac None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:44.056773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361258a-6591-448c-8a25-2a33b46a66ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:44.057663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361258a-6591-448c-8a25-2a33b46a66ee None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.057945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361258a-6591-448c-8a25-2a33b46a66ee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.058221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361258a-6591-448c-8a25-2a33b46a66ee None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/e7935e85cc3f4ed1ad5b01ba892e6e82/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.058848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd794bb-82bc-4b64-9ce6-5b39ebb138a4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:44.060291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361258a-6591-448c-8a25-2a33b46a66ee None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=e7935e85cc3f4ed1ad5b01ba892e6e82, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:44.062661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361258a-6591-448c-8a25-2a33b46a66ee None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:44.075236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:44.079534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 419/818] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:01:43 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/e7935e85cc3f4ed1ad5b01ba892e6e82/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:01:44.080578 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361258a-6591-448c-8a25-2a33b46a66ee None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:44.099666 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe971e05-0fb0-47ed-a76a-f94b4ce3eec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:44.100504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe971e05-0fb0-47ed-a76a-f94b4ce3eec7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.100895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe971e05-0fb0-47ed-a76a-f94b4ce3eec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.101305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe971e05-0fb0-47ed-a76a-f94b4ce3eec7 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.102384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe971e05-0fb0-47ed-a76a-f94b4ce3eec7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:44.103523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe971e05-0fb0-47ed-a76a-f94b4ce3eec7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:44.105113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe971e05-0fb0-47ed-a76a-f94b4ce3eec7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:44.107677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:44.137680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 400/819] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 22 11:01:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:44.143659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe971e05-0fb0-47ed-a76a-f94b4ce3eec7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:44.255561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe971e05-0fb0-47ed-a76a-f94b4ce3eec7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:44.307765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361258a-6591-448c-8a25-2a33b46a66ee None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:44.310716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f92d54-9550-4129-acd5-ac6dc1d35d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:44.311580 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f92d54-9550-4129-acd5-ac6dc1d35d93 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.312024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f92d54-9550-4129-acd5-ac6dc1d35d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.312408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f92d54-9550-4129-acd5-ac6dc1d35d93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.346539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1872a4-2f1c-442b-bf44-6c9b1b77c759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:44.347503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1872a4-2f1c-442b-bf44-6c9b1b77c759 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.347936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1872a4-2f1c-442b-bf44-6c9b1b77c759 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.348417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1872a4-2f1c-442b-bf44-6c9b1b77c759 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.348986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:44.350590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1872a4-2f1c-442b-bf44-6c9b1b77c759 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:44.352357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1872a4-2f1c-442b-bf44-6c9b1b77c759 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:44.376072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 420/820] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 22 11:01:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:44.377611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0fe3b3-2739-45bf-8926-f1f2318d2d70 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.377905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0fe3b3-2739-45bf-8926-f1f2318d2d70 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.378183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0fe3b3-2739-45bf-8926-f1f2318d2d70 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.386986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 421/821] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:44.392241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f92d54-9550-4129-acd5-ac6dc1d35d93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:44.393761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f92d54-9550-4129-acd5-ac6dc1d35d93 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:44.394378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05512934-d28c-4a82-9374-69055e2fa095 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.394875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05512934-d28c-4a82-9374-69055e2fa095 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.395454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05512934-d28c-4a82-9374-69055e2fa095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.416722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-05512934-d28c-4a82-9374-69055e2fa095 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:44.418493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-05512934-d28c-4a82-9374-69055e2fa095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:44.502687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 422/822] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:01:44.507293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:44.507467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 401/823] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:01:44.511044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05512934-d28c-4a82-9374-69055e2fa095 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:44.607370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f92d54-9550-4129-acd5-ac6dc1d35d93 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:44.657908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05512934-d28c-4a82-9374-69055e2fa095 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:44.676051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:44.695251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d80a01-03dc-464e-a731-a94f4ba5d4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:44.696083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d80a01-03dc-464e-a731-a94f4ba5d4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.696983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d80a01-03dc-464e-a731-a94f4ba5d4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.697283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d80a01-03dc-464e-a731-a94f4ba5d4d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.708144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d80a01-03dc-464e-a731-a94f4ba5d4d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:44.709600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d80a01-03dc-464e-a731-a94f4ba5d4d0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:44.723191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f92d54-9550-4129-acd5-ac6dc1d35d93 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-56d80a01-03dc-464e-a731-a94f4ba5d4d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:44.728945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:44.729834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:44.731149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:44.731149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:44.731149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:44.731149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:44.731149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 423/824] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:44.741840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d80a01-03dc-464e-a731-a94f4ba5d4d0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:44.769873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c61d47-30af-438e-847b-21e309697ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:44.770700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c61d47-30af-438e-847b-21e309697ecb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.770983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c61d47-30af-438e-847b-21e309697ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.771274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c61d47-30af-438e-847b-21e309697ecb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.771940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c61d47-30af-438e-847b-21e309697ecb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:44.773297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c61d47-30af-438e-847b-21e309697ecb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-06c61d47-30af-438e-847b-21e309697ecb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:44.791797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:44.792792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:44.793798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 402/825] 10.222.0.20 () {56 vars in 1129 bytes} [Fri Jul 22 11:01:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:44.801385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c61d47-30af-438e-847b-21e309697ecb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:44.814707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:44.871359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d80a01-03dc-464e-a731-a94f4ba5d4d0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:44.906330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81530d9d-4d29-4ead-af93-2e0293d4e654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:44.907104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81530d9d-4d29-4ead-af93-2e0293d4e654 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.907428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81530d9d-4d29-4ead-af93-2e0293d4e654 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.907713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81530d9d-4d29-4ead-af93-2e0293d4e654 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.908435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81530d9d-4d29-4ead-af93-2e0293d4e654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:44.908740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81530d9d-4d29-4ead-af93-2e0293d4e654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:44.910160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81530d9d-4d29-4ead-af93-2e0293d4e654 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:44.915439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c61d47-30af-438e-847b-21e309697ecb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:44.926731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 424/826] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:44.926731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e08347-0c67-496f-b080-fe7f9b0a165a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.926731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e08347-0c67-496f-b080-fe7f9b0a165a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.926731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e08347-0c67-496f-b080-fe7f9b0a165a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.926731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 425/827] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:44.930209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e08347-0c67-496f-b080-fe7f9b0a165a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:44.949221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:44.958342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc637ff-2abb-4997-b1fa-8972aa308b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:44.959162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc637ff-2abb-4997-b1fa-8972aa308b6a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.959481 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc637ff-2abb-4997-b1fa-8972aa308b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.959759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc637ff-2abb-4997-b1fa-8972aa308b6a None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.960459 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc637ff-2abb-4997-b1fa-8972aa308b6a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:44.960764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc637ff-2abb-4997-b1fa-8972aa308b6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:44.962155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc637ff-2abb-4997-b1fa-8972aa308b6a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:44.967197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 403/828] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 22 11:01:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:44.972687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ee4f7-eacb-471d-bae7-0e2ea915dcf3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:44.973061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ee4f7-eacb-471d-bae7-0e2ea915dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:44.973418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ee4f7-eacb-471d-bae7-0e2ea915dcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:44.991358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fa8ee4f7-eacb-471d-bae7-0e2ea915dcf3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:44.992659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fa8ee4f7-eacb-471d-bae7-0e2ea915dcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:45.044127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e08347-0c67-496f-b080-fe7f9b0a165a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:45.058637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:45.064761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 404/829] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:45.065411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ee4f7-eacb-471d-bae7-0e2ea915dcf3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:45.094892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ec98a-b709-4eb0-bcbb-286c0d42588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:45.095737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ec98a-b709-4eb0-bcbb-286c0d42588f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:45.096022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ec98a-b709-4eb0-bcbb-286c0d42588f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:45.096298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ec98a-b709-4eb0-bcbb-286c0d42588f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:45.103933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ec98a-b709-4eb0-bcbb-286c0d42588f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:45.105307 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ec98a-b709-4eb0-bcbb-286c0d42588f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-7c2ec98a-b709-4eb0-bcbb-286c0d42588f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:45.119994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:45.120880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 426/830] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:45.122496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ec98a-b709-4eb0-bcbb-286c0d42588f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:45.182799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ee4f7-eacb-471d-bae7-0e2ea915dcf3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:45.203377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:45.218790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938dbc5-530d-4e45-9b5a-014c495b05ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:45.219487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938dbc5-530d-4e45-9b5a-014c495b05ae None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:45.219803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938dbc5-530d-4e45-9b5a-014c495b05ae None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:45.220076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938dbc5-530d-4e45-9b5a-014c495b05ae None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:45.220790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938dbc5-530d-4e45-9b5a-014c495b05ae None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:45.221103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938dbc5-530d-4e45-9b5a-014c495b05ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:45.222483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938dbc5-530d-4e45-9b5a-014c495b05ae None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:45.227455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 405/831] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Jul 22 11:01:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:45.228728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938dbc5-530d-4e45-9b5a-014c495b05ae None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:45.235679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ec98a-b709-4eb0-bcbb-286c0d42588f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:45.276172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d545d46-9be7-44de-be16-6cf1c843d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:45.276969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d545d46-9be7-44de-be16-6cf1c843d999 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:45.277272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d545d46-9be7-44de-be16-6cf1c843d999 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:45.277535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d545d46-9be7-44de-be16-6cf1c843d999 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:45.288230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d545d46-9be7-44de-be16-6cf1c843d999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1d545d46-9be7-44de-be16-6cf1c843d999 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:45.296749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:45.297688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:01:45.298566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:45.302887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 427/832] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:01:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:45.303729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d545d46-9be7-44de-be16-6cf1c843d999 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:45.336271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938dbc5-530d-4e45-9b5a-014c495b05ae None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:45.336477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:45.372751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549c63d-fe7e-48d0-86a7-c60902b55a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:45.373329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549c63d-fe7e-48d0-86a7-c60902b55a55 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:45.373612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549c63d-fe7e-48d0-86a7-c60902b55a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:45.373890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549c63d-fe7e-48d0-86a7-c60902b55a55 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:45.374666 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549c63d-fe7e-48d0-86a7-c60902b55a55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:45.374978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549c63d-fe7e-48d0-86a7-c60902b55a55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:45.376456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549c63d-fe7e-48d0-86a7-c60902b55a55 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:45.388443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 406/833] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:45.389051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549c63d-fe7e-48d0-86a7-c60902b55a55 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:45.416770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d545d46-9be7-44de-be16-6cf1c843d999 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:45.456940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be266c0-2b93-4874-9822-9bc6db36b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:45.457629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be266c0-2b93-4874-9822-9bc6db36b048 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:45.457920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be266c0-2b93-4874-9822-9bc6db36b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:45.458202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be266c0-2b93-4874-9822-9bc6db36b048 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:45.458965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be266c0-2b93-4874-9822-9bc6db36b048 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:45.460414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be266c0-2b93-4874-9822-9bc6db36b048 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:45.491291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 428/834] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 22 11:01:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:45.491291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be266c0-2b93-4874-9822-9bc6db36b048 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:45.566953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:45.615478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:45.639525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be266c0-2b93-4874-9822-9bc6db36b048 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:45.640319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549c63d-fe7e-48d0-86a7-c60902b55a55 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:45.682553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae7007-3305-42e4-ab39-868206910222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:45.683221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae7007-3305-42e4-ab39-868206910222 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:45.683545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae7007-3305-42e4-ab39-868206910222 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:45.683829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae7007-3305-42e4-ab39-868206910222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:45.684665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae7007-3305-42e4-ab39-868206910222 None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=89224d740c7148938c2d803804083a57, user.id=c51929d0ecb047b4bdc405dc86fe7eb6) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:45.684991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae7007-3305-42e4-ab39-868206910222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:45.686428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae7007-3305-42e4-ab39-868206910222 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:45.691179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc1f39-17b6-4d3d-934c-b019cb3b6156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:45.692048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc1f39-17b6-4d3d-934c-b019cb3b6156 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:45.692333 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc1f39-17b6-4d3d-934c-b019cb3b6156 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:45.692607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc1f39-17b6-4d3d-934c-b019cb3b6156 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:45.693209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 429/835] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:45 2022] GET /identity/v3/role_assignments?user.id=c51929d0ecb047b4bdc405dc86fe7eb6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:45.693397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc1f39-17b6-4d3d-934c-b019cb3b6156 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:45.693711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc1f39-17b6-4d3d-934c-b019cb3b6156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:45.695074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc1f39-17b6-4d3d-934c-b019cb3b6156 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:45.705442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 407/836] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:01:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:45.711452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae7007-3305-42e4-ab39-868206910222 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:45.832884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae7007-3305-42e4-ab39-868206910222 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:45.870955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730772b-2ea5-4ed1-9c52-63f7634d1a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:45.871787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730772b-2ea5-4ed1-9c52-63f7634d1a5f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:45.872076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730772b-2ea5-4ed1-9c52-63f7634d1a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:45.872353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730772b-2ea5-4ed1-9c52-63f7634d1a5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:45.884940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730772b-2ea5-4ed1-9c52-63f7634d1a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:45.886392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730772b-2ea5-4ed1-9c52-63f7634d1a5f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:45.895520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8730772b-2ea5-4ed1-9c52-63f7634d1a5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:45.906477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:45.907601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:45.908676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:45.908676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:45.908676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:45.908676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:45.910861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 430/837] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:45.916969 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc1f39-17b6-4d3d-934c-b019cb3b6156 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:46.024022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc1f39-17b6-4d3d-934c-b019cb3b6156 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:46.032407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:46.061641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cedb3-d3a3-4ec3-a7b1-4ccd72a82f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:46.062729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cedb3-d3a3-4ec3-a7b1-4ccd72a82f5b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:46.063222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cedb3-d3a3-4ec3-a7b1-4ccd72a82f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:46.063731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cedb3-d3a3-4ec3-a7b1-4ccd72a82f5b None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:46.064662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cedb3-d3a3-4ec3-a7b1-4ccd72a82f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:46.065171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cedb3-d3a3-4ec3-a7b1-4ccd72a82f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:46.066766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cedb3-d3a3-4ec3-a7b1-4ccd72a82f5b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:46.080185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 408/838] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:46.087566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730772b-2ea5-4ed1-9c52-63f7634d1a5f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:46.193114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730772b-2ea5-4ed1-9c52-63f7634d1a5f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:46.228360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0ae8f-137b-41eb-a76d-acd7d6497d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:46.229180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0ae8f-137b-41eb-a76d-acd7d6497d7f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:46.229461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0ae8f-137b-41eb-a76d-acd7d6497d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:46.229735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0ae8f-137b-41eb-a76d-acd7d6497d7f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:46.237562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0ae8f-137b-41eb-a76d-acd7d6497d7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:46.239252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0ae8f-137b-41eb-a76d-acd7d6497d7f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e5d0ae8f-137b-41eb-a76d-acd7d6497d7f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:46.254800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:46.255820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:46.256771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 431/839] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:01:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:46.263134 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cedb3-d3a3-4ec3-a7b1-4ccd72a82f5b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:46.365472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cedb3-d3a3-4ec3-a7b1-4ccd72a82f5b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:46.394606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cde966-1787-47b6-8f11-bcd53ebd097f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:46.394908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cde966-1787-47b6-8f11-bcd53ebd097f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:46.395174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cde966-1787-47b6-8f11-bcd53ebd097f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:46.397766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 432/840] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:46.401386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433df92-f4ec-4c5e-b147-bf5befab64d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:46.401760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2fee9-c2fa-4f7f-a11c-007a6c2f2a85 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:46.402313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2fee9-c2fa-4f7f-a11c-007a6c2f2a85 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:46.402605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2fee9-c2fa-4f7f-a11c-007a6c2f2a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:46.403030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433df92-f4ec-4c5e-b147-bf5befab64d7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:46.403560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433df92-f4ec-4c5e-b147-bf5befab64d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:46.404053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433df92-f4ec-4c5e-b147-bf5befab64d7 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:46.405086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433df92-f4ec-4c5e-b147-bf5befab64d7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:46.405599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433df92-f4ec-4c5e-b147-bf5befab64d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:46.407218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433df92-f4ec-4c5e-b147-bf5befab64d7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:46.420442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5fc2fee9-c2fa-4f7f-a11c-007a6c2f2a85 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:46.422277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5fc2fee9-c2fa-4f7f-a11c-007a6c2f2a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:46.425472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 409/841] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 22 11:01:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:46.431609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433df92-f4ec-4c5e-b147-bf5befab64d7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:46.457701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:46.493903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:46.512266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 433/842] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:01:46.537407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2fee9-c2fa-4f7f-a11c-007a6c2f2a85 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:46.554375 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433df92-f4ec-4c5e-b147-bf5befab64d7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:46.590707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681060c0-298e-4d43-bc42-fe9d0a3e1977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:46.602350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681060c0-298e-4d43-bc42-fe9d0a3e1977 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:46.603436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681060c0-298e-4d43-bc42-fe9d0a3e1977 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:46.603936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681060c0-298e-4d43-bc42-fe9d0a3e1977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:46.652146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2fee9-c2fa-4f7f-a11c-007a6c2f2a85 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:46.681710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681060c0-298e-4d43-bc42-fe9d0a3e1977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:46.683364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681060c0-298e-4d43-bc42-fe9d0a3e1977 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:46.694391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2827c6-0463-46e4-943a-a14403ac2d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:46.695211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2827c6-0463-46e4-943a-a14403ac2d50 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:46.695529 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2827c6-0463-46e4-943a-a14403ac2d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:46.695808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2827c6-0463-46e4-943a-a14403ac2d50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:46.707481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2827c6-0463-46e4-943a-a14403ac2d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:46.711839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-dd2827c6-0463-46e4-943a-a14403ac2d50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:46.717460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:46.718733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:46.719961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 434/843] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:46.726613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2827c6-0463-46e4-943a-a14403ac2d50 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:46.789708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:46.814650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 410/844] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 22 11:01:46.817272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681060c0-298e-4d43-bc42-fe9d0a3e1977 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:46.853130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2827c6-0463-46e4-943a-a14403ac2d50 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:46.897875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e012de-1d65-4337-87c2-32d76da798e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:46.898349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e012de-1d65-4337-87c2-32d76da798e8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:46.898647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e012de-1d65-4337-87c2-32d76da798e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:46.898932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e012de-1d65-4337-87c2-32d76da798e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:46.899725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e012de-1d65-4337-87c2-32d76da798e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:46.900037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e012de-1d65-4337-87c2-32d76da798e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:46.901500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e012de-1d65-4337-87c2-32d76da798e8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:46.912673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 435/845] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:01:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:46.920911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e012de-1d65-4337-87c2-32d76da798e8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:46.949043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681060c0-298e-4d43-bc42-fe9d0a3e1977 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:46.982176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:46.999492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0ba43-27c4-4b0f-a9c6-a49d16864633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.000586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0ba43-27c4-4b0f-a9c6-a49d16864633 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.001019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0ba43-27c4-4b0f-a9c6-a49d16864633 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.001436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0ba43-27c4-4b0f-a9c6-a49d16864633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.016335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0ba43-27c4-4b0f-a9c6-a49d16864633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:47.018377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0ba43-27c4-4b0f-a9c6-a49d16864633 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:47.033185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e012de-1d65-4337-87c2-32d76da798e8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-16a0ba43-27c4-4b0f-a9c6-a49d16864633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:47.043436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:47.044495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:47.045578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:47.045578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:47.045578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:47.045578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:47.052175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 411/846] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:47.053809 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0ba43-27c4-4b0f-a9c6-a49d16864633 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:47.054506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:47.079858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad538a6-6983-41f0-99f2-dedaf89e9777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.080742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad538a6-6983-41f0-99f2-dedaf89e9777 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.081125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad538a6-6983-41f0-99f2-dedaf89e9777 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.081484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad538a6-6983-41f0-99f2-dedaf89e9777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.093257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad538a6-6983-41f0-99f2-dedaf89e9777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:47.094889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad538a6-6983-41f0-99f2-dedaf89e9777 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5ad538a6-6983-41f0-99f2-dedaf89e9777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:47.113178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:47.114364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:47.115594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:47.115594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:47.115594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:47.115594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:47.119572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 436/847] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:47.120556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad538a6-6983-41f0-99f2-dedaf89e9777 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:47.188597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0ba43-27c4-4b0f-a9c6-a49d16864633 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:47.231549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad538a6-6983-41f0-99f2-dedaf89e9777 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:47.240698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451c82b-42a3-4177-abed-bd943ee3aa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.242180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451c82b-42a3-4177-abed-bd943ee3aa53 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.242649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451c82b-42a3-4177-abed-bd943ee3aa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.243134 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451c82b-42a3-4177-abed-bd943ee3aa53 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.244523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451c82b-42a3-4177-abed-bd943ee3aa53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:47.245397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451c82b-42a3-4177-abed-bd943ee3aa53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:47.247283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451c82b-42a3-4177-abed-bd943ee3aa53 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:47.267967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 412/848] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:47.269569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451c82b-42a3-4177-abed-bd943ee3aa53 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:47.278553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf5195-ffd3-489f-84b7-7aeb623e28af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.279575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf5195-ffd3-489f-84b7-7aeb623e28af None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.280515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf5195-ffd3-489f-84b7-7aeb623e28af None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.280975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf5195-ffd3-489f-84b7-7aeb623e28af None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.282383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf5195-ffd3-489f-84b7-7aeb623e28af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:47.282881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf5195-ffd3-489f-84b7-7aeb623e28af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:47.285141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf5195-ffd3-489f-84b7-7aeb623e28af None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:47.302998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 437/849] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:47.310009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf5195-ffd3-489f-84b7-7aeb623e28af None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:47.311594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:47.342305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:47.387423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451c82b-42a3-4177-abed-bd943ee3aa53 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:47.423480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c760b8a-f583-4b8c-bb2f-b6f4a5544d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.424702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760b8a-f583-4b8c-bb2f-b6f4a5544d8f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.425227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760b8a-f583-4b8c-bb2f-b6f4a5544d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.425758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760b8a-f583-4b8c-bb2f-b6f4a5544d8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.433065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c760b8a-f583-4b8c-bb2f-b6f4a5544d8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:47.434763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c760b8a-f583-4b8c-bb2f-b6f4a5544d8f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:47.435556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf5195-ffd3-489f-84b7-7aeb623e28af None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0c760b8a-f583-4b8c-bb2f-b6f4a5544d8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:47.450329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:47.451288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:47.452304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:47.452304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:47.452304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:47.452304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:47.452304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:47.452304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:47.452304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:47.452304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:47.454783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 413/850] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:47.464216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c760b8a-f583-4b8c-bb2f-b6f4a5544d8f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:47.479024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb4e60-74f1-45fc-8df2-b49b923d275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.480130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb4e60-74f1-45fc-8df2-b49b923d275b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.480417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb4e60-74f1-45fc-8df2-b49b923d275b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.480697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb4e60-74f1-45fc-8df2-b49b923d275b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.489721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb4e60-74f1-45fc-8df2-b49b923d275b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:47.491969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb4e60-74f1-45fc-8df2-b49b923d275b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-57fb4e60-74f1-45fc-8df2-b49b923d275b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:47.510687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:47.511740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 22 11:01:47.512712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:47.515386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 438/851] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:01:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:47.518957 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb4e60-74f1-45fc-8df2-b49b923d275b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:47.578289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:47.584873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c760b8a-f583-4b8c-bb2f-b6f4a5544d8f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:47.612050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:47.632194 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb4e60-74f1-45fc-8df2-b49b923d275b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:47.634394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671e551-be05-43ee-af0b-5740154f6728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.635771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e551-be05-43ee-af0b-5740154f6728 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.636287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e551-be05-43ee-af0b-5740154f6728 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.636786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e551-be05-43ee-af0b-5740154f6728 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.638185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671e551-be05-43ee-af0b-5740154f6728 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:47.638498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671e551-be05-43ee-af0b-5740154f6728 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:47.639977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671e551-be05-43ee-af0b-5740154f6728 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:47.653072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 414/852] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:47.654539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cea7bd-e1f3-493c-8554-5ca18d6d2fb3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.655294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cea7bd-e1f3-493c-8554-5ca18d6d2fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.655294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cea7bd-e1f3-493c-8554-5ca18d6d2fb3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.656733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 415/853] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:47.662136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815ab43-b115-4c5c-8536-0b6f15f34b94 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.662850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815ab43-b115-4c5c-8536-0b6f15f34b94 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.663133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815ab43-b115-4c5c-8536-0b6f15f34b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.669304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6af96-eaac-41e9-94a3-c616ac6fded0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.670253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6af96-eaac-41e9-94a3-c616ac6fded0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.670365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6af96-eaac-41e9-94a3-c616ac6fded0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.670711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6af96-eaac-41e9-94a3-c616ac6fded0 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.671641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6af96-eaac-41e9-94a3-c616ac6fded0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:47.671960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6af96-eaac-41e9-94a3-c616ac6fded0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:47.673401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6af96-eaac-41e9-94a3-c616ac6fded0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:47.680042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1815ab43-b115-4c5c-8536-0b6f15f34b94 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:47.681791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1815ab43-b115-4c5c-8536-0b6f15f34b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:47.692592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 439/854] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Jul 22 11:01:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:47.693420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6af96-eaac-41e9-94a3-c616ac6fded0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:47.754803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 416/855] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:01:47.756100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815ab43-b115-4c5c-8536-0b6f15f34b94 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:47.803079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6af96-eaac-41e9-94a3-c616ac6fded0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:47.850222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9a170-d042-432f-a73d-965dbe11bd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.851945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9a170-d042-432f-a73d-965dbe11bd86 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.852503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9a170-d042-432f-a73d-965dbe11bd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.853738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9a170-d042-432f-a73d-965dbe11bd86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.854905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:47.855995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9a170-d042-432f-a73d-965dbe11bd86 None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=646d7c5e6c8e4825a7fe06d018ee319c) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:47.856731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9a170-d042-432f-a73d-965dbe11bd86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:47.859096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9a170-d042-432f-a73d-965dbe11bd86 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:47.875297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 440/856] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:47 2022] GET /identity/v3/role_assignments?user.id=646d7c5e6c8e4825a7fe06d018ee319c&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:47.875297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9a170-d042-432f-a73d-965dbe11bd86 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:47.877470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:47.883973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815ab43-b115-4c5c-8536-0b6f15f34b94 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:47.935804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a81f9-42b4-410f-bb35-7a3f4825e277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:47.936653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a81f9-42b4-410f-bb35-7a3f4825e277 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:47.937007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a81f9-42b4-410f-bb35-7a3f4825e277 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:47.937365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a81f9-42b4-410f-bb35-7a3f4825e277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:47.987351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9a170-d042-432f-a73d-965dbe11bd86 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.012452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a81f9-42b4-410f-bb35-7a3f4825e277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:48.013863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a81f9-42b4-410f-bb35-7a3f4825e277 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:48.026772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6328a-cd71-4169-993b-32dcb65007de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:48.027908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6328a-cd71-4169-993b-32dcb65007de None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.028192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6328a-cd71-4169-993b-32dcb65007de None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.028464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6328a-cd71-4169-993b-32dcb65007de None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.029118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6328a-cd71-4169-993b-32dcb65007de None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:48.030477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6328a-cd71-4169-993b-32dcb65007de None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ebd6328a-cd71-4169-993b-32dcb65007de None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:48.044707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:48.045749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 22 11:01:48.046602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:48.047637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 441/857] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:48.048374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7005c-ce97-461a-a692-18a0b3bedd2c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.049478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7005c-ce97-461a-a692-18a0b3bedd2c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.049761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7005c-ce97-461a-a692-18a0b3bedd2c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.055196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 442/858] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:48.056041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7005c-ce97-461a-a692-18a0b3bedd2c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:48.120956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:48.138903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 417/859] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:01:48.140195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce2c53-4fb4-4473-86b8-50069824f25d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.140492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce2c53-4fb4-4473-86b8-50069824f25d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.140771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce2c53-4fb4-4473-86b8-50069824f25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.145123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:48.157686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ddce2c53-4fb4-4473-86b8-50069824f25d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:48.158912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ddce2c53-4fb4-4473-86b8-50069824f25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:48.180422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7005c-ce97-461a-a692-18a0b3bedd2c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.216982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb002826-af23-47e6-898b-872221ef2743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:48.217645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb002826-af23-47e6-898b-872221ef2743 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.217929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb002826-af23-47e6-898b-872221ef2743 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.218205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb002826-af23-47e6-898b-872221ef2743 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.218955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb002826-af23-47e6-898b-872221ef2743 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:48.219270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb002826-af23-47e6-898b-872221ef2743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:48.220684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb002826-af23-47e6-898b-872221ef2743 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:48.227312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 443/860] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:48.227312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb002826-af23-47e6-898b-872221ef2743 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:48.233165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 418/861] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:48.234193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce2c53-4fb4-4473-86b8-50069824f25d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:48.341676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb002826-af23-47e6-898b-872221ef2743 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.348045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce2c53-4fb4-4473-86b8-50069824f25d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.364793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:48.377261 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a84a0-55d7-4e6b-89e2-6f2e3e0afa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:48.378067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a84a0-55d7-4e6b-89e2-6f2e3e0afa25 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.378356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a84a0-55d7-4e6b-89e2-6f2e3e0afa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.378636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a84a0-55d7-4e6b-89e2-6f2e3e0afa25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.380699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:48.383471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24404dd5-7ddb-4806-b217-2a036a2ee0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:48.384650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24404dd5-7ddb-4806-b217-2a036a2ee0fe None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.384934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24404dd5-7ddb-4806-b217-2a036a2ee0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.385211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24404dd5-7ddb-4806-b217-2a036a2ee0fe None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.385916 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24404dd5-7ddb-4806-b217-2a036a2ee0fe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:48.386222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24404dd5-7ddb-4806-b217-2a036a2ee0fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:48.387658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24404dd5-7ddb-4806-b217-2a036a2ee0fe None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:48.392222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 419/862] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:48.392519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a84a0-55d7-4e6b-89e2-6f2e3e0afa25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:48.393004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24404dd5-7ddb-4806-b217-2a036a2ee0fe None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:48.394352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a84a0-55d7-4e6b-89e2-6f2e3e0afa25 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-cf7a84a0-55d7-4e6b-89e2-6f2e3e0afa25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:48.423549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:48.424536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:48.425425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:48.425425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:48.425425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:48.425425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:48.426446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 444/863] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:48.432639 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a84a0-55d7-4e6b-89e2-6f2e3e0afa25 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:48.538431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24404dd5-7ddb-4806-b217-2a036a2ee0fe None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.565527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a84a0-55d7-4e6b-89e2-6f2e3e0afa25 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.580921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c5916-21e9-4535-a832-8cf20fc3ca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:48.581739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c5916-21e9-4535-a832-8cf20fc3ca25 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.582027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c5916-21e9-4535-a832-8cf20fc3ca25 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.582309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c5916-21e9-4535-a832-8cf20fc3ca25 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.583035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c5916-21e9-4535-a832-8cf20fc3ca25 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:48.584514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c5916-21e9-4535-a832-8cf20fc3ca25 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-472c5916-21e9-4535-a832-8cf20fc3ca25 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:48.600470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:48.601530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:48.602427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 420/864] 10.222.0.20 () {56 vars in 1132 bytes} [Fri Jul 22 11:01:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:48.608090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c5916-21e9-4535-a832-8cf20fc3ca25 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:48.623647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c2013-5054-4552-aa21-5dfed2f05569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:48.624650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c2013-5054-4552-aa21-5dfed2f05569 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.625144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c2013-5054-4552-aa21-5dfed2f05569 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.625761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c2013-5054-4552-aa21-5dfed2f05569 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.626692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c2013-5054-4552-aa21-5dfed2f05569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:48.627202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c2013-5054-4552-aa21-5dfed2f05569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:48.628919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c2013-5054-4552-aa21-5dfed2f05569 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:48.647399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 445/865] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:48.648790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c2013-5054-4552-aa21-5dfed2f05569 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:48.677135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:48.708659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:48.760179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c5916-21e9-4535-a832-8cf20fc3ca25 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.770601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c2013-5054-4552-aa21-5dfed2f05569 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.804994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d16a2-0cfc-4ab9-8abe-80f1862a8ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:48.806665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d16a2-0cfc-4ab9-8abe-80f1862a8ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.807050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d16a2-0cfc-4ab9-8abe-80f1862a8ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.812551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d16a2-0cfc-4ab9-8abe-80f1862a8ef0 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.812551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d16a2-0cfc-4ab9-8abe-80f1862a8ef0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:48.812551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d16a2-0cfc-4ab9-8abe-80f1862a8ef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:48.812551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d16a2-0cfc-4ab9-8abe-80f1862a8ef0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:48.815356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9319a3-a5a1-430c-a340-9d26e423c46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:48.816753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9319a3-a5a1-430c-a340-9d26e423c46e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.817252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9319a3-a5a1-430c-a340-9d26e423c46e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.817755 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9319a3-a5a1-430c-a340-9d26e423c46e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.822535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 421/866] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:01:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:48.823428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d16a2-0cfc-4ab9-8abe-80f1862a8ef0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:48.829106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9319a3-a5a1-430c-a340-9d26e423c46e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:48.830820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9319a3-a5a1-430c-a340-9d26e423c46e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ff9319a3-a5a1-430c-a340-9d26e423c46e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:48.847220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:48.848271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:48.849265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:48.849265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:48.849265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:48.849265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:48.849265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:48.849265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:48.849265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:48.849265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:48.857340 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 446/867] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:48.858795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9319a3-a5a1-430c-a340-9d26e423c46e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:48.946551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d16a2-0cfc-4ab9-8abe-80f1862a8ef0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.959515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:48.989676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9319a3-a5a1-430c-a340-9d26e423c46e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:48.995354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a9a46-b25a-470d-9edc-33e392e0cfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:48.996329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a9a46-b25a-470d-9edc-33e392e0cfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:48.996780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a9a46-b25a-470d-9edc-33e392e0cfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:48.997331 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a9a46-b25a-470d-9edc-33e392e0cfe4 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:48.998825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a9a46-b25a-470d-9edc-33e392e0cfe4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:48.999343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a9a46-b25a-470d-9edc-33e392e0cfe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:49.000765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:49.001898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a9a46-b25a-470d-9edc-33e392e0cfe4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:49.015684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 422/868] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:01:48 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:49.035866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4030f3e-992f-433f-8cd0-0da99438f8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:49.036975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4030f3e-992f-433f-8cd0-0da99438f8df None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:49.037325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4030f3e-992f-433f-8cd0-0da99438f8df None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:49.037699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4030f3e-992f-433f-8cd0-0da99438f8df None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:49.038836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4030f3e-992f-433f-8cd0-0da99438f8df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:49.039530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4030f3e-992f-433f-8cd0-0da99438f8df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:49.041995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4030f3e-992f-433f-8cd0-0da99438f8df None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:49.057027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 447/869] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:49.063035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a9a46-b25a-470d-9edc-33e392e0cfe4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:49.185335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a9a46-b25a-470d-9edc-33e392e0cfe4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:49.225615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcbcba-1a09-43a1-8e49-457a540bfbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:49.226592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcbcba-1a09-43a1-8e49-457a540bfbac None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:49.227075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcbcba-1a09-43a1-8e49-457a540bfbac None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:49.228163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcbcba-1a09-43a1-8e49-457a540bfbac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:49.229194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcbcba-1a09-43a1-8e49-457a540bfbac None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=89224d740c7148938c2d803804083a57, user.id=e7935e85cc3f4ed1ad5b01ba892e6e82) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:49.229700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcbcba-1a09-43a1-8e49-457a540bfbac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:49.231366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcbcba-1a09-43a1-8e49-457a540bfbac None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:49.238412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 423/870] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:49 2022] GET /identity/v3/role_assignments?user.id=e7935e85cc3f4ed1ad5b01ba892e6e82&scope.project.id=89224d740c7148938c2d803804083a57&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:50.015565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6da6-905e-4f27-bd17-0829adf09cfd None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:50.016063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6da6-905e-4f27-bd17-0829adf09cfd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:50.016063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6da6-905e-4f27-bd17-0829adf09cfd None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:50.017648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 448/871] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:50.022803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12272ad9-5575-482b-a507-8ed92b86ec71 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:50.023239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12272ad9-5575-482b-a507-8ed92b86ec71 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:50.023455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12272ad9-5575-482b-a507-8ed92b86ec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:50.041256 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-12272ad9-5575-482b-a507-8ed92b86ec71 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:50.042438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-12272ad9-5575-482b-a507-8ed92b86ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:50.072423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:50.117724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 424/872] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:50.140807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e6da6-905e-4f27-bd17-0829adf09cfd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:50.252845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e6da6-905e-4f27-bd17-0829adf09cfd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:50.289241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a24a5-e9fe-4025-a39a-81c193a00f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:50.290605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a24a5-e9fe-4025-a39a-81c193a00f3b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:50.290898 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a24a5-e9fe-4025-a39a-81c193a00f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:50.291178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a24a5-e9fe-4025-a39a-81c193a00f3b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:50.301941 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a24a5-e9fe-4025-a39a-81c193a00f3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-dd8a24a5-e9fe-4025-a39a-81c193a00f3b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:50.310135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:50.311160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:50.312061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 449/873] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 22 11:01:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:50.317243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12272ad9-5575-482b-a507-8ed92b86ec71 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:50.418549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12272ad9-5575-482b-a507-8ed92b86ec71 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:50.459191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d905d4d-0ee7-463d-af02-6a55e34a3dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:50.459911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d905d4d-0ee7-463d-af02-6a55e34a3dbd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:50.460207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d905d4d-0ee7-463d-af02-6a55e34a3dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:50.460482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d905d4d-0ee7-463d-af02-6a55e34a3dbd None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:50.461240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d905d4d-0ee7-463d-af02-6a55e34a3dbd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:50.461547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d905d4d-0ee7-463d-af02-6a55e34a3dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:50.463015 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d905d4d-0ee7-463d-af02-6a55e34a3dbd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:50.471924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 425/874] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 22 11:01:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:50.477084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a24a5-e9fe-4025-a39a-81c193a00f3b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:50.506478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:50.583293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a24a5-e9fe-4025-a39a-81c193a00f3b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:50.611917 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e9e55-7727-4212-aec9-bf57a308b740 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:50.612343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e9e55-7727-4212-aec9-bf57a308b740 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:50.612729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e9e55-7727-4212-aec9-bf57a308b740 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:50.614694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 426/875] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:50.620244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91f707-1ca6-4b65-913c-438f5899ea7f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:50.620751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91f707-1ca6-4b65-913c-438f5899ea7f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:50.621231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91f707-1ca6-4b65-913c-438f5899ea7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:50.622103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f6a73-789b-48bb-bb2b-54ef3b0fea52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:50.622945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f6a73-789b-48bb-bb2b-54ef3b0fea52 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:50.623284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f6a73-789b-48bb-bb2b-54ef3b0fea52 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:50.623577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f6a73-789b-48bb-bb2b-54ef3b0fea52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:50.635043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f6a73-789b-48bb-bb2b-54ef3b0fea52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:50.636583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f6a73-789b-48bb-bb2b-54ef3b0fea52 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:50.639239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8d91f707-1ca6-4b65-913c-438f5899ea7f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:50.640693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8d91f707-1ca6-4b65-913c-438f5899ea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-477f6a73-789b-48bb-bb2b-54ef3b0fea52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:50.655057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:50.656036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:50.656946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:50.656946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:50.656946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:50.656946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:50.656946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 450/876] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:50.662074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f6a73-789b-48bb-bb2b-54ef3b0fea52 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:50.667630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:50.716076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 427/877] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:50.739753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91f707-1ca6-4b65-913c-438f5899ea7f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:50.745685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:50.770805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f6a73-789b-48bb-bb2b-54ef3b0fea52 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:50.807044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c90e1-ba4b-4915-8e40-d16b0d96926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:50.808021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c90e1-ba4b-4915-8e40-d16b0d96926d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:50.808471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c90e1-ba4b-4915-8e40-d16b0d96926d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:50.808902 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c90e1-ba4b-4915-8e40-d16b0d96926d None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:50.809786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c90e1-ba4b-4915-8e40-d16b0d96926d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:50.810300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c90e1-ba4b-4915-8e40-d16b0d96926d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:50.811890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c90e1-ba4b-4915-8e40-d16b0d96926d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:50.829044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 451/878] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:50.830134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c90e1-ba4b-4915-8e40-d16b0d96926d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:50.844269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91f707-1ca6-4b65-913c-438f5899ea7f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:50.880847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383c768-e701-4f4e-ab64-418c2a7bda09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:50.881524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383c768-e701-4f4e-ab64-418c2a7bda09 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:50.881812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383c768-e701-4f4e-ab64-418c2a7bda09 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:50.882089 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383c768-e701-4f4e-ab64-418c2a7bda09 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:50.882813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383c768-e701-4f4e-ab64-418c2a7bda09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:50.884257 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383c768-e701-4f4e-ab64-418c2a7bda09 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:50.911285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 428/879] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:01:50 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:01:50.950016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c90e1-ba4b-4915-8e40-d16b0d96926d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:50.999137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eabb9c-fc03-4a99-8a72-a29bb6da8c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:51.000069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabb9c-fc03-4a99-8a72-a29bb6da8c9f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.000362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabb9c-fc03-4a99-8a72-a29bb6da8c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.000643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabb9c-fc03-4a99-8a72-a29bb6da8c9f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.002766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:51.009813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eabb9c-fc03-4a99-8a72-a29bb6da8c9f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:51.011405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eabb9c-fc03-4a99-8a72-a29bb6da8c9f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e2eabb9c-fc03-4a99-8a72-a29bb6da8c9f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:51.029012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:51.030452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:51.031449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 452/880] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:01:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:51.036451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383c768-e701-4f4e-ab64-418c2a7bda09 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:51.059715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:51.148733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383c768-e701-4f4e-ab64-418c2a7bda09 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:51.185136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df25ae6-6a23-4087-b07f-cc9c13f394b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:51.185825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df25ae6-6a23-4087-b07f-cc9c13f394b6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.186115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df25ae6-6a23-4087-b07f-cc9c13f394b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.186397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df25ae6-6a23-4087-b07f-cc9c13f394b6 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.188287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c9795-4a03-4638-8c2a-db7ae4259304 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.188579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c9795-4a03-4638-8c2a-db7ae4259304 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.188857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c9795-4a03-4638-8c2a-db7ae4259304 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.190389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 453/881] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:51.190768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df25ae6-6a23-4087-b07f-cc9c13f394b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:51.191083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df25ae6-6a23-4087-b07f-cc9c13f394b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:51.192557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df25ae6-6a23-4087-b07f-cc9c13f394b6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:51.196972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e9a22-7d4e-45f7-9163-bc45885a872d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.197262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e9a22-7d4e-45f7-9163-bc45885a872d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.197543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e9a22-7d4e-45f7-9163-bc45885a872d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.214048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 429/882] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 22 11:01:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:51.215357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1b8e9a22-7d4e-45f7-9163-bc45885a872d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:51.216798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1b8e9a22-7d4e-45f7-9163-bc45885a872d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:51.221094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df25ae6-6a23-4087-b07f-cc9c13f394b6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:51.292610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 454/883] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:51.309822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:51.315748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e9a22-7d4e-45f7-9163-bc45885a872d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:51.341125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df25ae6-6a23-4087-b07f-cc9c13f394b6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:51.379156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaebf1-6939-45b5-be99-ce278e0354ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:51.380283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaebf1-6939-45b5-be99-ce278e0354ef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.380819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaebf1-6939-45b5-be99-ce278e0354ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.381314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaebf1-6939-45b5-be99-ce278e0354ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.438868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e9a22-7d4e-45f7-9163-bc45885a872d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:51.456172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefaebf1-6939-45b5-be99-ce278e0354ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:51.458097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefaebf1-6939-45b5-be99-ce278e0354ef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:51.464779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:51.478022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bc035-1074-43d1-8f63-37001739685e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:51.478697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc035-1074-43d1-8f63-37001739685e None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.478984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc035-1074-43d1-8f63-37001739685e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.479270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc035-1074-43d1-8f63-37001739685e None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.480030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bc035-1074-43d1-8f63-37001739685e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:51.481403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bc035-1074-43d1-8f63-37001739685e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:51.497415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 455/884] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:01:51 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:01:51.498568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2b3c9-adfd-43a6-a9aa-5f0fdbb47e3d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.498862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2b3c9-adfd-43a6-a9aa-5f0fdbb47e3d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.499141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2b3c9-adfd-43a6-a9aa-5f0fdbb47e3d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.500616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 456/885] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:51.506101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e1831-bb3b-49bb-b327-9a6ff6f9ee89 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.506389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e1831-bb3b-49bb-b327-9a6ff6f9ee89 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.506677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e1831-bb3b-49bb-b327-9a6ff6f9ee89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.529884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c63e1831-bb3b-49bb-b327-9a6ff6f9ee89 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:51.531316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c63e1831-bb3b-49bb-b327-9a6ff6f9ee89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:51.556285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:51.584169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 430/886] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:01:51 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:01:51.596778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaebf1-6939-45b5-be99-ce278e0354ef None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:51.634586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 457/887] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:01:51.653045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e1831-bb3b-49bb-b327-9a6ff6f9ee89 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:51.707394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaebf1-6939-45b5-be99-ce278e0354ef None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:51.743876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe7912-d9f1-4df6-8336-660f03b96869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:51.744764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe7912-d9f1-4df6-8336-660f03b96869 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.745052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe7912-d9f1-4df6-8336-660f03b96869 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.745360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe7912-d9f1-4df6-8336-660f03b96869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.756244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe7912-d9f1-4df6-8336-660f03b96869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:51.757711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe7912-d9f1-4df6-8336-660f03b96869 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:51.762392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e1831-bb3b-49bb-b327-9a6ff6f9ee89 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:51.773662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1cbe7912-d9f1-4df6-8336-660f03b96869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:51.777679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:51.778793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:51.779863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:51.779863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:51.779863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:51.779863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:51.779863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 431/888] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:51.786118 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe7912-d9f1-4df6-8336-660f03b96869 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:51.801456 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03b6d8-207b-422a-ac06-94d21ccb4d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:51.802271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03b6d8-207b-422a-ac06-94d21ccb4d0e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.802559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03b6d8-207b-422a-ac06-94d21ccb4d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.802995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03b6d8-207b-422a-ac06-94d21ccb4d0e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.803629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03b6d8-207b-422a-ac06-94d21ccb4d0e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:51.805094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03b6d8-207b-422a-ac06-94d21ccb4d0e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:51.818006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5f03b6d8-207b-422a-ac06-94d21ccb4d0e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:51.819741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:51.821010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:01:51.822054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:51.824596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 458/889] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:01:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:51.830437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03b6d8-207b-422a-ac06-94d21ccb4d0e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:51.911849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe7912-d9f1-4df6-8336-660f03b96869 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:51.955174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff75d92-9e01-43ea-a0d3-caf7237f5a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:51.955891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03b6d8-207b-422a-ac06-94d21ccb4d0e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:51.956306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff75d92-9e01-43ea-a0d3-caf7237f5a79 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:51.956753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff75d92-9e01-43ea-a0d3-caf7237f5a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:51.957191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff75d92-9e01-43ea-a0d3-caf7237f5a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:51.958339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff75d92-9e01-43ea-a0d3-caf7237f5a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:51.959544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff75d92-9e01-43ea-a0d3-caf7237f5a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:51.963980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff75d92-9e01-43ea-a0d3-caf7237f5a79 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:51.979056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 432/890] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:51.984231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff75d92-9e01-43ea-a0d3-caf7237f5a79 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:52.006829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b58921-765f-4977-9988-91365206e6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:52.007706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b58921-765f-4977-9988-91365206e6ee None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:52.007997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b58921-765f-4977-9988-91365206e6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:52.008286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b58921-765f-4977-9988-91365206e6ee None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:52.009328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b58921-765f-4977-9988-91365206e6ee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:52.009648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b58921-765f-4977-9988-91365206e6ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:52.011030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b58921-765f-4977-9988-91365206e6ee None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:52.021280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 459/891] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:01:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:52.023389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b58921-765f-4977-9988-91365206e6ee None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:52.074606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:52.099469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:52.115944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff75d92-9e01-43ea-a0d3-caf7237f5a79 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:52.143103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b58921-765f-4977-9988-91365206e6ee None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:52.155867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe05fa6-f9fa-4fc8-a92c-ed6f0d90a54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:52.157080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe05fa6-f9fa-4fc8-a92c-ed6f0d90a54f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:52.157666 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe05fa6-f9fa-4fc8-a92c-ed6f0d90a54f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:52.158151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe05fa6-f9fa-4fc8-a92c-ed6f0d90a54f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:52.166331 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe05fa6-f9fa-4fc8-a92c-ed6f0d90a54f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:52.168303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe05fa6-f9fa-4fc8-a92c-ed6f0d90a54f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:52.178161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f1326-8cd5-47ee-994b-f3c123e57657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:52.178828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f1326-8cd5-47ee-994b-f3c123e57657 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:52.179116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f1326-8cd5-47ee-994b-f3c123e57657 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:52.179430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f1326-8cd5-47ee-994b-f3c123e57657 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:52.180141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f1326-8cd5-47ee-994b-f3c123e57657 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:52.181001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f1326-8cd5-47ee-994b-f3c123e57657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:52.182362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f1326-8cd5-47ee-994b-f3c123e57657 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0fe05fa6-f9fa-4fc8-a92c-ed6f0d90a54f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:52.185794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:52.186669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:52.187607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:52.187607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:52.187607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:52.187607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:52.187607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:52.187607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:52.187607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:52.187607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:52.190128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 433/892] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:52.198427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe05fa6-f9fa-4fc8-a92c-ed6f0d90a54f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:52.198628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 460/893] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:01:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:52.312772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe05fa6-f9fa-4fc8-a92c-ed6f0d90a54f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:52.350606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb23a5-39b8-4bf8-8869-05ecfa318e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:52.351664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb23a5-39b8-4bf8-8869-05ecfa318e8b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:52.352328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb23a5-39b8-4bf8-8869-05ecfa318e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:52.352619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb23a5-39b8-4bf8-8869-05ecfa318e8b None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:52.353405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb23a5-39b8-4bf8-8869-05ecfa318e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:52.353716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb23a5-39b8-4bf8-8869-05ecfa318e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:52.355144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb23a5-39b8-4bf8-8869-05ecfa318e8b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:52.367452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 434/894] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:52.373574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f1326-8cd5-47ee-994b-f3c123e57657 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:52.477179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f1326-8cd5-47ee-994b-f3c123e57657 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:52.478731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:52.513733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312e156-9752-4e9d-9977-8b68d460528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:52.514443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312e156-9752-4e9d-9977-8b68d460528a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:52.514778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312e156-9752-4e9d-9977-8b68d460528a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:52.515062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312e156-9752-4e9d-9977-8b68d460528a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:52.515925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312e156-9752-4e9d-9977-8b68d460528a None admin] RBAC: Adding query filter params (role.id=19b08ccf876549a4b4f59fd06c217412, scope.project.id=89224d740c7148938c2d803804083a57, user.id=646d7c5e6c8e4825a7fe06d018ee319c) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:52.516236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312e156-9752-4e9d-9977-8b68d460528a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:52.517671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312e156-9752-4e9d-9977-8b68d460528a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:52.523736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 461/895] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:01:52 2022] GET /identity/v3/role_assignments?user.id=646d7c5e6c8e4825a7fe06d018ee319c&scope.project.id=89224d740c7148938c2d803804083a57&role.id=19b08ccf876549a4b4f59fd06c217412 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:53.066091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c4e26-7723-426b-8bb3-00c379c6874e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:53.066463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c4e26-7723-426b-8bb3-00c379c6874e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:53.066658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c4e26-7723-426b-8bb3-00c379c6874e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:53.068293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 435/896] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:53.074155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcc1b7-caea-411a-b3ee-ff14c2016db5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:53.074468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcc1b7-caea-411a-b3ee-ff14c2016db5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:53.074730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcc1b7-caea-411a-b3ee-ff14c2016db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:53.091419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-dffcc1b7-caea-411a-b3ee-ff14c2016db5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:53.092595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dffcc1b7-caea-411a-b3ee-ff14c2016db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:53.162726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 462/897] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:53.185376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c4e26-7723-426b-8bb3-00c379c6874e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:53.197654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:53.292183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c4e26-7723-426b-8bb3-00c379c6874e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:53.326668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49663a2-61b5-44c4-879b-6f770236f1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:53.327439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49663a2-61b5-44c4-879b-6f770236f1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:53.327861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49663a2-61b5-44c4-879b-6f770236f1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:53.328135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49663a2-61b5-44c4-879b-6f770236f1c4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:53.328794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49663a2-61b5-44c4-879b-6f770236f1c4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:53.330218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49663a2-61b5-44c4-879b-6f770236f1c4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e49663a2-61b5-44c4-879b-6f770236f1c4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:53.342359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:53.343367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:53.344226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 436/898] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:53.349705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcc1b7-caea-411a-b3ee-ff14c2016db5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:53.448527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:53.455699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcc1b7-caea-411a-b3ee-ff14c2016db5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:53.490517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794626d8-4b63-452b-a22b-0c1c8479a6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:53.491187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794626d8-4b63-452b-a22b-0c1c8479a6bd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:53.491505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794626d8-4b63-452b-a22b-0c1c8479a6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:53.491782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794626d8-4b63-452b-a22b-0c1c8479a6bd None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:53.492486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794626d8-4b63-452b-a22b-0c1c8479a6bd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:53.492834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794626d8-4b63-452b-a22b-0c1c8479a6bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:53.494195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794626d8-4b63-452b-a22b-0c1c8479a6bd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:53.499358 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 463/899] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:53.505033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49663a2-61b5-44c4-879b-6f770236f1c4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:53.578104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:53.609367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49663a2-61b5-44c4-879b-6f770236f1c4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:53.643370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47ef92-7d0c-4131-b43b-d1c821b62b1c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:53.643685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47ef92-7d0c-4131-b43b-d1c821b62b1c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:53.644040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47ef92-7d0c-4131-b43b-d1c821b62b1c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:53.645748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 464/900] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:53.647088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3c5b5-cd2a-4ab3-b529-b0171b09009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:53.647856 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3c5b5-cd2a-4ab3-b529-b0171b09009e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:53.648160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3c5b5-cd2a-4ab3-b529-b0171b09009e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:53.648446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3c5b5-cd2a-4ab3-b529-b0171b09009e None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:53.649744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3c5b5-cd2a-4ab3-b529-b0171b09009e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:53.650060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3c5b5-cd2a-4ab3-b529-b0171b09009e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:53.654575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3c5b5-cd2a-4ab3-b529-b0171b09009e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:53.656584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a64a2-254d-45ab-ad21-248c1e7c4d04 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:53.657185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a64a2-254d-45ab-ad21-248c1e7c4d04 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:53.657677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a64a2-254d-45ab-ad21-248c1e7c4d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:53.659305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 437/901] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:53.665063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3c5b5-cd2a-4ab3-b529-b0171b09009e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:53.682253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bb4a64a2-254d-45ab-ad21-248c1e7c4d04 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:53.683616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bb4a64a2-254d-45ab-ad21-248c1e7c4d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:53.777000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 465/902] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:53.787099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3c5b5-cd2a-4ab3-b529-b0171b09009e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:53.798442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a64a2-254d-45ab-ad21-248c1e7c4d04 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:53.823085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53afd8f-b855-4c9f-89fe-b9d5ea506517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:53.823778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53afd8f-b855-4c9f-89fe-b9d5ea506517 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:53.824069 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53afd8f-b855-4c9f-89fe-b9d5ea506517 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:53.824350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53afd8f-b855-4c9f-89fe-b9d5ea506517 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:53.825157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53afd8f-b855-4c9f-89fe-b9d5ea506517 None admin] RBAC: Adding query filter params (interface=public, service_id=d03dcdf007cf494aa78a144dd1e90edb, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:53.825452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53afd8f-b855-4c9f-89fe-b9d5ea506517 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:53.826825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53afd8f-b855-4c9f-89fe-b9d5ea506517 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:53.831478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:53.835779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 438/903] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 22 11:01:53 2022] GET /identity/v3/endpoints?service_id=d03dcdf007cf494aa78a144dd1e90edb&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:53.844689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53afd8f-b855-4c9f-89fe-b9d5ea506517 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:53.899983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:53.912974 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a64a2-254d-45ab-ad21-248c1e7c4d04 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:53.951941 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c5de-9124-44ec-a189-8502a0a8f539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:53.953155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c5de-9124-44ec-a189-8502a0a8f539 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:53.953650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c5de-9124-44ec-a189-8502a0a8f539 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:53.954133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c5de-9124-44ec-a189-8502a0a8f539 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:53.955004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026c5de-9124-44ec-a189-8502a0a8f539 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:53.956654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026c5de-9124-44ec-a189-8502a0a8f539 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:53.968214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53afd8f-b855-4c9f-89fe-b9d5ea506517 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8026c5de-9124-44ec-a189-8502a0a8f539 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:53.970284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:53.971238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 22 11:01:53.972378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:53.975110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 466/904] 10.222.0.20 () {56 vars in 1132 bytes} [Fri Jul 22 11:01:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:53.980836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c5de-9124-44ec-a189-8502a0a8f539 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:54.012868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7663c-2fb8-4822-a18f-1626a98111f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:54.013545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7663c-2fb8-4822-a18f-1626a98111f8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.013834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7663c-2fb8-4822-a18f-1626a98111f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.014111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7663c-2fb8-4822-a18f-1626a98111f8 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.014802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7663c-2fb8-4822-a18f-1626a98111f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:54.016203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7663c-2fb8-4822-a18f-1626a98111f8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:54.027567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 439/905] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 22 11:01:53 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:01:54.103629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c5de-9124-44ec-a189-8502a0a8f539 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:54.146616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cac57-fb36-4299-826d-d2b1227b504f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:54.147755 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cac57-fb36-4299-826d-d2b1227b504f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.148256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cac57-fb36-4299-826d-d2b1227b504f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.148756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cac57-fb36-4299-826d-d2b1227b504f None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.149772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cac57-fb36-4299-826d-d2b1227b504f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:54.150301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cac57-fb36-4299-826d-d2b1227b504f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:54.151983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cac57-fb36-4299-826d-d2b1227b504f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:54.158405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 467/906] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:01:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:54.163896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7663c-2fb8-4822-a18f-1626a98111f8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:54.232462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:54.274345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7663c-2fb8-4822-a18f-1626a98111f8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:54.319811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd24b30-9baa-42cb-b4c8-66ef8b051f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:54.320487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd24b30-9baa-42cb-b4c8-66ef8b051f19 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.320819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd24b30-9baa-42cb-b4c8-66ef8b051f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.321124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd24b30-9baa-42cb-b4c8-66ef8b051f19 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.321835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd24b30-9baa-42cb-b4c8-66ef8b051f19 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:54.322205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd24b30-9baa-42cb-b4c8-66ef8b051f19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:54.323605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd24b30-9baa-42cb-b4c8-66ef8b051f19 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:54.329393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 440/907] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:01:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:54.335196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cac57-fb36-4299-826d-d2b1227b504f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:54.357003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:54.410360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d311c5a-e52a-43bc-8ba5-0311b30871d7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.410718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d311c5a-e52a-43bc-8ba5-0311b30871d7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.410987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d311c5a-e52a-43bc-8ba5-0311b30871d7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.412610 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 441/908] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:54.418200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a7390-bf09-4cbc-b3ae-3446f931b87e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.418490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a7390-bf09-4cbc-b3ae-3446f931b87e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.419538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a7390-bf09-4cbc-b3ae-3446f931b87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.436192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f8a7390-bf09-4cbc-b3ae-3446f931b87e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:54.437389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f8a7390-bf09-4cbc-b3ae-3446f931b87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:54.455668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cac57-fb36-4299-826d-d2b1227b504f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:54.496961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b7ecd-b3a9-4ec8-92cb-1bf57e96fcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:54.497919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b7ecd-b3a9-4ec8-92cb-1bf57e96fcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.498466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b7ecd-b3a9-4ec8-92cb-1bf57e96fcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.498906 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b7ecd-b3a9-4ec8-92cb-1bf57e96fcd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.499995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b7ecd-b3a9-4ec8-92cb-1bf57e96fcd6 None admin] RBAC: Adding query filter params (interface=public, service_id=f1510ac7bc504b73b331561cdd19a52a, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:54.500464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b7ecd-b3a9-4ec8-92cb-1bf57e96fcd6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:54.502190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b7ecd-b3a9-4ec8-92cb-1bf57e96fcd6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:54.508350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 468/909] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 22 11:01:54 2022] GET /identity/v3/endpoints?service_id=f1510ac7bc504b73b331561cdd19a52a&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:54.513643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b7ecd-b3a9-4ec8-92cb-1bf57e96fcd6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:54.534052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 442/910] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:01:54.556243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a7390-bf09-4cbc-b3ae-3446f931b87e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:54.625466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:54.636580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:54.661389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b7ecd-b3a9-4ec8-92cb-1bf57e96fcd6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:54.685009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a7390-bf09-4cbc-b3ae-3446f931b87e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:54.697237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f6b7a-1042-433a-83f7-5095f3768a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:54.697804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f6b7a-1042-433a-83f7-5095f3768a22 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.698094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f6b7a-1042-433a-83f7-5095f3768a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.698378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f6b7a-1042-433a-83f7-5095f3768a22 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.699067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f6b7a-1042-433a-83f7-5095f3768a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:54.700491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f6b7a-1042-433a-83f7-5095f3768a22 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:54.713657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 469/911] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 22 11:01:54 2022] GET /identity/v3/services => generated 1268 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:01:54.713657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e9c0-4a31-40cc-a162-e62746794c60 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.713657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e9c0-4a31-40cc-a162-e62746794c60 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.713657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e9c0-4a31-40cc-a162-e62746794c60 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.713657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 470/912] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:54.715439 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a12f-6a39-41ea-99f3-d50ab4f7d507 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.715743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a12f-6a39-41ea-99f3-d50ab4f7d507 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.716023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a12f-6a39-41ea-99f3-d50ab4f7d507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.732043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6c8a4-4292-4910-843b-5bfa2d94f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:54.732823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6c8a4-4292-4910-843b-5bfa2d94f8cf None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:54.733183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6c8a4-4292-4910-843b-5bfa2d94f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:54.733572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6c8a4-4292-4910-843b-5bfa2d94f8cf None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:54.734326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6c8a4-4292-4910-843b-5bfa2d94f8cf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:54.734430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-23e0a12f-6a39-41ea-99f3-d50ab4f7d507 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:54.735594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-23e0a12f-6a39-41ea-99f3-d50ab4f7d507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:54.735807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6c8a4-4292-4910-843b-5bfa2d94f8cf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:54.752315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 443/913] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:01:54 2022] POST /identity/v3/services => generated 241 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:54.829100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 471/914] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:01:54.853027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6c8a4-4292-4910-843b-5bfa2d94f8cf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:54.967211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6c8a4-4292-4910-843b-5bfa2d94f8cf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:54.985301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:55.006724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4648d5-00ac-4fd2-91e2-70697c6dd1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:55.008030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4648d5-00ac-4fd2-91e2-70697c6dd1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:55.008030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4648d5-00ac-4fd2-91e2-70697c6dd1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:55.008220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4648d5-00ac-4fd2-91e2-70697c6dd1b5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:55.024889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4648d5-00ac-4fd2-91e2-70697c6dd1b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ff4648d5-00ac-4fd2-91e2-70697c6dd1b5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:55.033088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:55.033994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:55.034838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 444/915] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 22 11:01:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:55.040281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0a12f-6a39-41ea-99f3-d50ab4f7d507 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:55.111452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:55.144299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0a12f-6a39-41ea-99f3-d50ab4f7d507 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:55.179613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a428e1-3690-4e2c-8597-380ed9b19ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:55.180868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a428e1-3690-4e2c-8597-380ed9b19ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:55.181152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a428e1-3690-4e2c-8597-380ed9b19ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:55.181426 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a428e1-3690-4e2c-8597-380ed9b19ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:55.182233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a428e1-3690-4e2c-8597-380ed9b19ab8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:55.182538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a428e1-3690-4e2c-8597-380ed9b19ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:55.183928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a428e1-3690-4e2c-8597-380ed9b19ab8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:55.192564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 472/916] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 22 11:01:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:55.198404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4648d5-00ac-4fd2-91e2-70697c6dd1b5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:55.310583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4648d5-00ac-4fd2-91e2-70697c6dd1b5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:55.345137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b6f1-1dc0-4b4b-a485-04d21d517a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:55.345933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b6f1-1dc0-4b4b-a485-04d21d517a95 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:55.346216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b6f1-1dc0-4b4b-a485-04d21d517a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:55.346490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b6f1-1dc0-4b4b-a485-04d21d517a95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:55.361638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447b6f1-1dc0-4b4b-a485-04d21d517a95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:55.363383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447b6f1-1dc0-4b4b-a485-04d21d517a95 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1447b6f1-1dc0-4b4b-a485-04d21d517a95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:55.381632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:55.382530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:55.384203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:55.384203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:55.384203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:55.384203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:55.384203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 445/917] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:55.389685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a428e1-3690-4e2c-8597-380ed9b19ab8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:55.490187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a428e1-3690-4e2c-8597-380ed9b19ab8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:55.528951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64443395-de08-4d18-88b0-2677077fae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:55.529625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64443395-de08-4d18-88b0-2677077fae72 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:55.529913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64443395-de08-4d18-88b0-2677077fae72 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:55.530189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64443395-de08-4d18-88b0-2677077fae72 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:55.530905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64443395-de08-4d18-88b0-2677077fae72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:55.531212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64443395-de08-4d18-88b0-2677077fae72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:55.532594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64443395-de08-4d18-88b0-2677077fae72 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:55.534081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:55.550915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 473/918] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:55.551855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b6f1-1dc0-4b4b-a485-04d21d517a95 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:55.585200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:55.663401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b6f1-1dc0-4b4b-a485-04d21d517a95 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:55.698863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce93509-d712-4435-9bc2-c12c4620aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:55.699732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce93509-d712-4435-9bc2-c12c4620aed0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:55.700075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce93509-d712-4435-9bc2-c12c4620aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:55.700358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce93509-d712-4435-9bc2-c12c4620aed0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:55.708102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce93509-d712-4435-9bc2-c12c4620aed0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:55.709562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce93509-d712-4435-9bc2-c12c4620aed0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3ce93509-d712-4435-9bc2-c12c4620aed0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:01:55.724352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:55.725471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:55.726424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 446/919] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:01:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:55.731911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64443395-de08-4d18-88b0-2677077fae72 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:55.832689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64443395-de08-4d18-88b0-2677077fae72 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:55.868262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3493d6b-cb22-4d76-bd2c-7da7d4bd0365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:55.868802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3493d6b-cb22-4d76-bd2c-7da7d4bd0365 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:55.869083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3493d6b-cb22-4d76-bd2c-7da7d4bd0365 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:55.869354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3493d6b-cb22-4d76-bd2c-7da7d4bd0365 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:55.870194 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3493d6b-cb22-4d76-bd2c-7da7d4bd0365 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:55.870501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3493d6b-cb22-4d76-bd2c-7da7d4bd0365 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:55.871958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3493d6b-cb22-4d76-bd2c-7da7d4bd0365 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:55.889464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 474/920] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 22 11:01:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:55.895189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce93509-d712-4435-9bc2-c12c4620aed0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:55.995798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:56.006479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce93509-d712-4435-9bc2-c12c4620aed0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:56.044821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225df7f-2b81-4abb-809b-24d59f478d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:56.045627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225df7f-2b81-4abb-809b-24d59f478d7c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.045916 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225df7f-2b81-4abb-809b-24d59f478d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.046201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225df7f-2b81-4abb-809b-24d59f478d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.057191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225df7f-2b81-4abb-809b-24d59f478d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:56.058734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225df7f-2b81-4abb-809b-24d59f478d7c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7225df7f-2b81-4abb-809b-24d59f478d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:56.076544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:56.077502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:56.078377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:56.078377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:56.078377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:56.078377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:56.078377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 447/921] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:56.084329 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3493d6b-cb22-4d76-bd2c-7da7d4bd0365 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:56.140917 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:56.185589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21b340-fcf0-45ea-b0fb-47744a8b40ef None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.185931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21b340-fcf0-45ea-b0fb-47744a8b40ef None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.186285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21b340-fcf0-45ea-b0fb-47744a8b40ef None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.187857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 448/922] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:01:56.189963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3493d6b-cb22-4d76-bd2c-7da7d4bd0365 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:56.196194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e27b5-06e4-4f95-85a3-f00eb4eab76d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.196705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e27b5-06e4-4f95-85a3-f00eb4eab76d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.197208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e27b5-06e4-4f95-85a3-f00eb4eab76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.219630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-029e27b5-06e4-4f95-85a3-f00eb4eab76d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:56.222011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-029e27b5-06e4-4f95-85a3-f00eb4eab76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:56.229228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361dcd2-a5c7-487f-9bfa-7fb33a998b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:56.229899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361dcd2-a5c7-487f-9bfa-7fb33a998b2d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.230184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361dcd2-a5c7-487f-9bfa-7fb33a998b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.230464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361dcd2-a5c7-487f-9bfa-7fb33a998b2d None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.231178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361dcd2-a5c7-487f-9bfa-7fb33a998b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:56.231528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361dcd2-a5c7-487f-9bfa-7fb33a998b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:56.232997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361dcd2-a5c7-487f-9bfa-7fb33a998b2d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:56.250292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 475/923] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:56.251141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361dcd2-a5c7-487f-9bfa-7fb33a998b2d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:56.314576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 449/924] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:56.346041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e27b5-06e4-4f95-85a3-f00eb4eab76d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:56.372664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361dcd2-a5c7-487f-9bfa-7fb33a998b2d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:56.394690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:56.426524 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddc412-5ca2-4851-91da-2b4c4c46ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:56.427485 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddc412-5ca2-4851-91da-2b4c4c46ccfa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.427837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddc412-5ca2-4851-91da-2b4c4c46ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.428178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddc412-5ca2-4851-91da-2b4c4c46ccfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.433774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:56.437208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddc412-5ca2-4851-91da-2b4c4c46ccfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:56.438885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddc412-5ca2-4851-91da-2b4c4c46ccfa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-97ddc412-5ca2-4851-91da-2b4c4c46ccfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:56.456351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:56.457276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:56.458178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:56.458178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:56.458178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:56.458178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:56.458178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:56.458178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:56.458178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:56.458178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:56.465992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 476/925] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:56.466265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e27b5-06e4-4f95-85a3-f00eb4eab76d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:56.467427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddc412-5ca2-4851-91da-2b4c4c46ccfa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:56.503507 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b345822-d428-40bd-a07f-3cd0f4d9f495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:56.504321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b345822-d428-40bd-a07f-3cd0f4d9f495 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.504644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b345822-d428-40bd-a07f-3cd0f4d9f495 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.505620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b345822-d428-40bd-a07f-3cd0f4d9f495 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.505741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b345822-d428-40bd-a07f-3cd0f4d9f495 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:56.507045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b345822-d428-40bd-a07f-3cd0f4d9f495 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9b345822-d428-40bd-a07f-3cd0f4d9f495 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:56.520587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:56.521620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 22 11:01:56.522484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:56.527444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 450/926] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:56.528498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b345822-d428-40bd-a07f-3cd0f4d9f495 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:56.580758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddc412-5ca2-4851-91da-2b4c4c46ccfa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:56.624360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9db69-b816-42d5-97d8-d750ca7ab131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:56.625400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9db69-b816-42d5-97d8-d750ca7ab131 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.625983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9db69-b816-42d5-97d8-d750ca7ab131 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.626481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9db69-b816-42d5-97d8-d750ca7ab131 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.627504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9db69-b816-42d5-97d8-d750ca7ab131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:56.628026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9db69-b816-42d5-97d8-d750ca7ab131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:56.629921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9db69-b816-42d5-97d8-d750ca7ab131 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:56.639179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b345822-d428-40bd-a07f-3cd0f4d9f495 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:56.649548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 477/927] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:56.650759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9db69-b816-42d5-97d8-d750ca7ab131 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:56.678904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5e82e-5f62-48ee-b8ed-bfb098157107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:56.679931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5e82e-5f62-48ee-b8ed-bfb098157107 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.680884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5e82e-5f62-48ee-b8ed-bfb098157107 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.681301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5e82e-5f62-48ee-b8ed-bfb098157107 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.682512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5e82e-5f62-48ee-b8ed-bfb098157107 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:56.682987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5e82e-5f62-48ee-b8ed-bfb098157107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:56.685273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5e82e-5f62-48ee-b8ed-bfb098157107 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:56.686799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:56.689640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:56.703480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 451/928] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:56.706503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5e82e-5f62-48ee-b8ed-bfb098157107 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:56.772167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9db69-b816-42d5-97d8-d750ca7ab131 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:56.811112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cc0b0-65f7-4f40-9a46-ec3eac7bd055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:56.812530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cc0b0-65f7-4f40-9a46-ec3eac7bd055 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.813031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cc0b0-65f7-4f40-9a46-ec3eac7bd055 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.813506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cc0b0-65f7-4f40-9a46-ec3eac7bd055 None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/646d7c5e6c8e4825a7fe06d018ee319c/roles/19b08ccf876549a4b4f59fd06c217412` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.816281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cc0b0-65f7-4f40-9a46-ec3eac7bd055 None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=646d7c5e6c8e4825a7fe06d018ee319c, role_id=19b08ccf876549a4b4f59fd06c217412)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:56.817929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cc0b0-65f7-4f40-9a46-ec3eac7bd055 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:56.843547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5e82e-5f62-48ee-b8ed-bfb098157107 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:56.843753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 478/929] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:01:56 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/646d7c5e6c8e4825a7fe06d018ee319c/roles/19b08ccf876549a4b4f59fd06c217412 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:01:56.880771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9da5a-c337-4d87-9e20-d59b6cec9aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:56.881464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9da5a-c337-4d87-9e20-d59b6cec9aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.881752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9da5a-c337-4d87-9e20-d59b6cec9aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.882037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9da5a-c337-4d87-9e20-d59b6cec9aa7 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.882762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9da5a-c337-4d87-9e20-d59b6cec9aa7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:56.883074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9da5a-c337-4d87-9e20-d59b6cec9aa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:56.885125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9da5a-c337-4d87-9e20-d59b6cec9aa7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:56.891530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 452/930] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Jul 22 11:01:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:56.897244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cc0b0-65f7-4f40-9a46-ec3eac7bd055 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:56.936765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526b075-94ff-40bc-8f67-46cc379ea166 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.937047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526b075-94ff-40bc-8f67-46cc379ea166 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.937249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526b075-94ff-40bc-8f67-46cc379ea166 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.938639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 453/931] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:01:56.944377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9898336-bcb6-4275-88cb-b3cf3d4627ed None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:56.944686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9898336-bcb6-4275-88cb-b3cf3d4627ed None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:56.944952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9898336-bcb6-4275-88cb-b3cf3d4627ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:56.964296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a9898336-bcb6-4275-88cb-b3cf3d4627ed None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:56.965542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9898336-bcb6-4275-88cb-b3cf3d4627ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:57.023083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:57.038438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cc0b0-65f7-4f40-9a46-ec3eac7bd055 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:57.042924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:57.056374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 454/932] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:01:57.057949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc9dde-a5b7-4f2c-ae75-be520a99f585 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.058266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc9dde-a5b7-4f2c-ae75-be520a99f585 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.058524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc9dde-a5b7-4f2c-ae75-be520a99f585 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.060362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 455/933] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:57.068579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f296e0-c131-487f-8633-fa580c7f6fdf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.068579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f296e0-c131-487f-8633-fa580c7f6fdf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.068579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f296e0-c131-487f-8633-fa580c7f6fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.087636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f2145-b28a-48f9-aa78-435e7ec0a2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:57.088668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f2145-b28a-48f9-aa78-435e7ec0a2f2 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.089146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f2145-b28a-48f9-aa78-435e7ec0a2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.089765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f2145-b28a-48f9-aa78-435e7ec0a2f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.090712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f2145-b28a-48f9-aa78-435e7ec0a2f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:57.091559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e4f296e0-c131-487f-8633-fa580c7f6fdf None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:57.092305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f2145-b28a-48f9-aa78-435e7ec0a2f2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:57.092767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e4f296e0-c131-487f-8633-fa580c7f6fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:57.115978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 479/934] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 22 11:01:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:01:57.116689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f2145-b28a-48f9-aa78-435e7ec0a2f2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:57.207421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 456/935] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:57.226327 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f296e0-c131-487f-8633-fa580c7f6fdf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:57.259909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f2145-b28a-48f9-aa78-435e7ec0a2f2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:57.298564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646787d0-550d-4582-933e-ab9510d4fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:57.299853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646787d0-550d-4582-933e-ab9510d4fefa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.300137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646787d0-550d-4582-933e-ab9510d4fefa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.300505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646787d0-550d-4582-933e-ab9510d4fefa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.301057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646787d0-550d-4582-933e-ab9510d4fefa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:57.302425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646787d0-550d-4582-933e-ab9510d4fefa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-646787d0-550d-4582-933e-ab9510d4fefa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:57.315620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:57.316647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:57.317786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 480/936] 10.222.0.20 () {56 vars in 1132 bytes} [Fri Jul 22 11:01:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:57.322993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646787d0-550d-4582-933e-ab9510d4fefa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:57.342815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:57.346104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f296e0-c131-487f-8633-fa580c7f6fdf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:57.376313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:57.404692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870302e-2fe8-44f9-b20a-f2c3665683ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:57.405988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870302e-2fe8-44f9-b20a-f2c3665683ce None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.406410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870302e-2fe8-44f9-b20a-f2c3665683ce None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.406823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870302e-2fe8-44f9-b20a-f2c3665683ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.407522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870302e-2fe8-44f9-b20a-f2c3665683ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:57.409846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870302e-2fe8-44f9-b20a-f2c3665683ce None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7870302e-2fe8-44f9-b20a-f2c3665683ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:57.437460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:57.438545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:01:57.439561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:57.440662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 457/937] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:01:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:57.447352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870302e-2fe8-44f9-b20a-f2c3665683ce None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:57.464611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646787d0-550d-4582-933e-ab9510d4fefa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:57.502211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fa475-2472-4b9e-96d6-113aaced7169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:57.502886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fa475-2472-4b9e-96d6-113aaced7169 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.503312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fa475-2472-4b9e-96d6-113aaced7169 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.503609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fa475-2472-4b9e-96d6-113aaced7169 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.504321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fa475-2472-4b9e-96d6-113aaced7169 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:57.504633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fa475-2472-4b9e-96d6-113aaced7169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:57.506005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fa475-2472-4b9e-96d6-113aaced7169 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:57.514517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 481/938] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:01:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:57.520224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fa475-2472-4b9e-96d6-113aaced7169 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:57.578643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870302e-2fe8-44f9-b20a-f2c3665683ce None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:57.621371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:57.641866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e9b2-c165-4fc9-a5e4-20cd2ddfa542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:57.642777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e9b2-c165-4fc9-a5e4-20cd2ddfa542 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.643063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e9b2-c165-4fc9-a5e4-20cd2ddfa542 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.643371 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e9b2-c165-4fc9-a5e4-20cd2ddfa542 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.644393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023e9b2-c165-4fc9-a5e4-20cd2ddfa542 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:57.644753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023e9b2-c165-4fc9-a5e4-20cd2ddfa542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:57.646781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023e9b2-c165-4fc9-a5e4-20cd2ddfa542 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:57.648618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fa475-2472-4b9e-96d6-113aaced7169 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:57.659115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 458/939] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:01:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:57.660068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e9b2-c165-4fc9-a5e4-20cd2ddfa542 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:57.688971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbdac5-7989-461f-8fe9-370ff4578214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:57.690117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbdac5-7989-461f-8fe9-370ff4578214 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.690614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbdac5-7989-461f-8fe9-370ff4578214 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.691111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbdac5-7989-461f-8fe9-370ff4578214 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.692052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbdac5-7989-461f-8fe9-370ff4578214 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:57.692581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbdac5-7989-461f-8fe9-370ff4578214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:57.694223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbdac5-7989-461f-8fe9-370ff4578214 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:57.705086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 482/940] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:01:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:57.706540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbdac5-7989-461f-8fe9-370ff4578214 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:57.723126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:57.796385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e9b2-c165-4fc9-a5e4-20cd2ddfa542 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:57.835276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbdac5-7989-461f-8fe9-370ff4578214 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:57.836076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935dc16-5c4f-4017-a225-241d7c63c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:57.836999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935dc16-5c4f-4017-a225-241d7c63c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.837491 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935dc16-5c4f-4017-a225-241d7c63c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.837993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935dc16-5c4f-4017-a225-241d7c63c6c1 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.838925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935dc16-5c4f-4017-a225-241d7c63c6c1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:57.839479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935dc16-5c4f-4017-a225-241d7c63c6c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:57.841066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935dc16-5c4f-4017-a225-241d7c63c6c1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:57.846740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 459/941] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:01:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:57.852593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935dc16-5c4f-4017-a225-241d7c63c6c1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:57.882585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06574422-71e1-4382-81fd-768d5f16b501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:57.883838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06574422-71e1-4382-81fd-768d5f16b501 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:57.884574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06574422-71e1-4382-81fd-768d5f16b501 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:57.885489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06574422-71e1-4382-81fd-768d5f16b501 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:57.886706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06574422-71e1-4382-81fd-768d5f16b501 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:57.888991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06574422-71e1-4382-81fd-768d5f16b501 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:57.901511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:57.925310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 483/942] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 22 11:01:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:01:57.987709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935dc16-5c4f-4017-a225-241d7c63c6c1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:58.003135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:58.032834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85275e98-4df2-4e49-bb0b-fa50b94dc64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:58.033509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85275e98-4df2-4e49-bb0b-fa50b94dc64d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:58.033797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85275e98-4df2-4e49-bb0b-fa50b94dc64d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:58.034076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85275e98-4df2-4e49-bb0b-fa50b94dc64d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:58.034876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85275e98-4df2-4e49-bb0b-fa50b94dc64d None admin] RBAC: Adding query filter params (interface=public, service_id=50211284bb164417913688903755edf7, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:58.035189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85275e98-4df2-4e49-bb0b-fa50b94dc64d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:58.036607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85275e98-4df2-4e49-bb0b-fa50b94dc64d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:58.043295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 460/943] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 22 11:01:57 2022] GET /identity/v3/endpoints?service_id=50211284bb164417913688903755edf7&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:01:58.048893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06574422-71e1-4382-81fd-768d5f16b501 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:58.178149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06574422-71e1-4382-81fd-768d5f16b501 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:58.214348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32472333-5d9e-4436-b2c7-5bf10ad1a5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:58.215022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32472333-5d9e-4436-b2c7-5bf10ad1a5ab None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:58.215366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32472333-5d9e-4436-b2c7-5bf10ad1a5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:58.215655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32472333-5d9e-4436-b2c7-5bf10ad1a5ab None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:58.216343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32472333-5d9e-4436-b2c7-5bf10ad1a5ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:58.217889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32472333-5d9e-4436-b2c7-5bf10ad1a5ab None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:58.227791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 484/944] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 22 11:01:58 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:01:59.023818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df5636-de0f-4b3c-9fde-bf74c65e991f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.024308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df5636-de0f-4b3c-9fde-bf74c65e991f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.024308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df5636-de0f-4b3c-9fde-bf74c65e991f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.025798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 461/945] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:59.031838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8f952-6d71-417b-b760-0dc718e50025 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.032319 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8f952-6d71-417b-b760-0dc718e50025 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.032783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8f952-6d71-417b-b760-0dc718e50025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.051212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-eaf8f952-6d71-417b-b760-0dc718e50025 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:59.052459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eaf8f952-6d71-417b-b760-0dc718e50025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:59.111095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:59.123921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 485/946] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:59.148449 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df5636-de0f-4b3c-9fde-bf74c65e991f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:59.262219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df5636-de0f-4b3c-9fde-bf74c65e991f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:59.297103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9fd7a-9e6e-49bd-96dd-802d8492d6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:59.298267 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9fd7a-9e6e-49bd-96dd-802d8492d6da None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.298759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9fd7a-9e6e-49bd-96dd-802d8492d6da None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.299240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9fd7a-9e6e-49bd-96dd-802d8492d6da None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.310945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9fd7a-9e6e-49bd-96dd-802d8492d6da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:01:59.312191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90db015-f720-4bd2-8142-85b4b0887ade None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.312493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90db015-f720-4bd2-8142-85b4b0887ade None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.312787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90db015-f720-4bd2-8142-85b4b0887ade None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.314403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 486/947] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:01:59.320034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abd43-ddca-4321-ae0a-128a96568be2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.320331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abd43-ddca-4321-ae0a-128a96568be2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.320653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abd43-ddca-4321-ae0a-128a96568be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5be9fd7a-9e6e-49bd-96dd-802d8492d6da None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:01:59.321114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:59.322018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:01:59.322858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:59.324699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 462/948] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 22 11:01:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:59.330848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9fd7a-9e6e-49bd-96dd-802d8492d6da None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:59.342558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-dc8abd43-ddca-4321-ae0a-128a96568be2 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:01:59.343911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dc8abd43-ddca-4321-ae0a-128a96568be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:01:59.365702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:59.437114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 487/949] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:01:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:01:59.444168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9fd7a-9e6e-49bd-96dd-802d8492d6da None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:59.460158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8abd43-ddca-4321-ae0a-128a96568be2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:59.485475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab348970-96d6-4ea1-a525-4ab5a5dc01b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:59.486500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab348970-96d6-4ea1-a525-4ab5a5dc01b3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.487001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab348970-96d6-4ea1-a525-4ab5a5dc01b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.487527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab348970-96d6-4ea1-a525-4ab5a5dc01b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.489002 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab348970-96d6-4ea1-a525-4ab5a5dc01b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:59.489546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab348970-96d6-4ea1-a525-4ab5a5dc01b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:59.491204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab348970-96d6-4ea1-a525-4ab5a5dc01b3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:59.501983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 463/950] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 22 11:01:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:59.509482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab348970-96d6-4ea1-a525-4ab5a5dc01b3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:59.582601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8abd43-ddca-4321-ae0a-128a96568be2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:59.608370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:59.614155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:59.621718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab348970-96d6-4ea1-a525-4ab5a5dc01b3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:59.627837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:59.628742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.629249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.629713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.630599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:59.632242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:59.633882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING keystone.server.flask.common [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 22 11:01:59.652130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.sql.core [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 22 11:01:59.652130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 22 11:01:59.652130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [parameters: {'id': 'd4ae52de812248b3a99d4062db9f1177', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 22 11:01:59.652130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80393) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 22 11:01:59.660489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8788d-a9a4-429f-8cc1-f44f601dca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d4ae52de812248b3a99d4062db9f1177.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d4ae52de812248b3a99d4062db9f1177. Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result = self._query(query) Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application conn.query(q) Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 22 11:01:59.660659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result.read() Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 22 11:01:59.661453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application next(self.gen) Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application yield resource Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application next(self.gen) Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.session.rollback() Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.force_reraise() Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise self.value Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application session.commit() Jul 22 11:01:59.662415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._prepare_impl() Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.session.flush() Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._flush(objects) Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application compat.raise_( Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application flush_context.execute() Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 22 11:01:59.663351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rec.execute(self) Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 22 11:01:59.664914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result = self._query(query) Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application conn.query(q) Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result.read() Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 22 11:01:59.665727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [parameters: {'id': 'd4ae52de812248b3a99d4062db9f1177', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:59.666613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:59.668152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d4ae52de812248b3a99d4062db9f1177. Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:59.669417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 488/951] 10.222.0.20 () {60 vars in 1140 bytes} [Fri Jul 22 11:01:59 2022] POST /identity/v3/projects => generated 262 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:59.671017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8788d-a9a4-429f-8cc1-f44f601dca2f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.671399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8788d-a9a4-429f-8cc1-f44f601dca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.672195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8788d-a9a4-429f-8cc1-f44f601dca2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.672823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:59.688793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8788d-a9a4-429f-8cc1-f44f601dca2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:59.690720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8788d-a9a4-429f-8cc1-f44f601dca2f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a8c8788d-a9a4-429f-8cc1-f44f601dca2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:01:59.715404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:59.716366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:59.717277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:01:59.717277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:01:59.717277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:01:59.717277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:01:59.722287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 464/952] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:01:59.723232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8788d-a9a4-429f-8cc1-f44f601dca2f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:59.790636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa1833-e834-459f-a602-811a306b001b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:59.827515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73475c23-db3f-4067-a156-1e41062ec93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:59.828618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73475c23-db3f-4067-a156-1e41062ec93c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.829131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73475c23-db3f-4067-a156-1e41062ec93c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.829593 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73475c23-db3f-4067-a156-1e41062ec93c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.837553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73475c23-db3f-4067-a156-1e41062ec93c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:59.839448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73475c23-db3f-4067-a156-1e41062ec93c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-73475c23-db3f-4067-a156-1e41062ec93c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:01:59.854890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:01:59.856025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:01:59.856981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:01:59.856981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:01:59.856981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:01:59.856981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:01:59.856981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:01:59.856981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:01:59.856981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:01:59.856981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:01:59.857409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8788d-a9a4-429f-8cc1-f44f601dca2f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:01:59.859674 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 489/953] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:01:59.868314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73475c23-db3f-4067-a156-1e41062ec93c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:59.904136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbce149-5aff-43b6-bd1f-2477f8b68ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:01:59.905524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbce149-5aff-43b6-bd1f-2477f8b68ecd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:01:59.906037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbce149-5aff-43b6-bd1f-2477f8b68ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:01:59.906541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbce149-5aff-43b6-bd1f-2477f8b68ecd None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:01:59.907648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbce149-5aff-43b6-bd1f-2477f8b68ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:01:59.907905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:01:59.908397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbce149-5aff-43b6-bd1f-2477f8b68ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:01:59.910806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbce149-5aff-43b6-bd1f-2477f8b68ecd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:01:59.931573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 465/954] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:01:59.932633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbce149-5aff-43b6-bd1f-2477f8b68ecd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:01:59.946950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:00.001084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73475c23-db3f-4067-a156-1e41062ec93c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:00.051615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d017c6-dc81-4356-9b74-6d66fa7151e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:00.053156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d017c6-dc81-4356-9b74-6d66fa7151e7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.053606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbce149-5aff-43b6-bd1f-2477f8b68ecd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:00.054186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d017c6-dc81-4356-9b74-6d66fa7151e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.054676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d017c6-dc81-4356-9b74-6d66fa7151e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.056854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d017c6-dc81-4356-9b74-6d66fa7151e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:00.057411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d017c6-dc81-4356-9b74-6d66fa7151e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:00.059105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d017c6-dc81-4356-9b74-6d66fa7151e7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:00.095471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 490/955] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:01:59 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:00.097311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b270eb-314f-4211-a650-24b9a6d7246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:00.098158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b270eb-314f-4211-a650-24b9a6d7246f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.098444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b270eb-314f-4211-a650-24b9a6d7246f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.098724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b270eb-314f-4211-a650-24b9a6d7246f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.107535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b270eb-314f-4211-a650-24b9a6d7246f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:00.110048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0997c6-566f-4c6e-bdbc-d45c108d015d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.110330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0997c6-566f-4c6e-bdbc-d45c108d015d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.110618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0997c6-566f-4c6e-bdbc-d45c108d015d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.111793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b270eb-314f-4211-a650-24b9a6d7246f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:00.112460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 491/956] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:02:00.118716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcbadb-437b-4e76-abf5-22d663ea88f7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.119334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcbadb-437b-4e76-abf5-22d663ea88f7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.119827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcbadb-437b-4e76-abf5-22d663ea88f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e0b270eb-314f-4211-a650-24b9a6d7246f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:02:00.132396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:00.133993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 22 11:02:00.135035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:00.136106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 466/957] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:01:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:00.141728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b270eb-314f-4211-a650-24b9a6d7246f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:00.143963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-adbcbadb-437b-4e76-abf5-22d663ea88f7 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:00.145542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-adbcbadb-437b-4e76-abf5-22d663ea88f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:00.227605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:00.243286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 492/958] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:02:00.259881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcbadb-437b-4e76-abf5-22d663ea88f7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:00.278931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b270eb-314f-4211-a650-24b9a6d7246f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:00.294809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:00.319862 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a026-408d-4a87-a1c0-444bc94f2913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:00.321170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a026-408d-4a87-a1c0-444bc94f2913 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.321531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a026-408d-4a87-a1c0-444bc94f2913 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.321883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a026-408d-4a87-a1c0-444bc94f2913 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.322812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a026-408d-4a87-a1c0-444bc94f2913 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:00.323157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a026-408d-4a87-a1c0-444bc94f2913 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:00.325530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a026-408d-4a87-a1c0-444bc94f2913 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:00.346940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 467/959] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Jul 22 11:02:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:00.348686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4de423-2887-465d-8a17-6cb8dd8cf80d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.349151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4de423-2887-465d-8a17-6cb8dd8cf80d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.349615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4de423-2887-465d-8a17-6cb8dd8cf80d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.351316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 468/960] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:00.353121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4de423-2887-465d-8a17-6cb8dd8cf80d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:00.391733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcbadb-437b-4e76-abf5-22d663ea88f7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:00.427580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f771e2-3e7c-4e39-941e-cfbe35ed1764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:00.428444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f771e2-3e7c-4e39-941e-cfbe35ed1764 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.428741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f771e2-3e7c-4e39-941e-cfbe35ed1764 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.429023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f771e2-3e7c-4e39-941e-cfbe35ed1764 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.429678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f771e2-3e7c-4e39-941e-cfbe35ed1764 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:00.431766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f771e2-3e7c-4e39-941e-cfbe35ed1764 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f4f771e2-3e7c-4e39-941e-cfbe35ed1764 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:00.444450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:00.446151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:00.447043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 493/961] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:02:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:00.447888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a09ff-41e4-4933-85f6-df6ec35a891c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.447888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a09ff-41e4-4933-85f6-df6ec35a891c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.448719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a09ff-41e4-4933-85f6-df6ec35a891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.465553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4f9a09ff-41e4-4933-85f6-df6ec35a891c None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:00.466853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4f9a09ff-41e4-4933-85f6-df6ec35a891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:00.469562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4de423-2887-465d-8a17-6cb8dd8cf80d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:00.494876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:00.507785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c60ad-fabc-449c-8888-e279fdd1bc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:00.509080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c60ad-fabc-449c-8888-e279fdd1bc0d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.509465 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c60ad-fabc-449c-8888-e279fdd1bc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.509706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c60ad-fabc-449c-8888-e279fdd1bc0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.540068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 494/962] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:00.543496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a09ff-41e4-4933-85f6-df6ec35a891c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:00.573214 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:00.584612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c60ad-fabc-449c-8888-e279fdd1bc0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:00.586102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c60ad-fabc-449c-8888-e279fdd1bc0d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:00.664277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a09ff-41e4-4933-85f6-df6ec35a891c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:00.701711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c634e44-2cc5-4214-b300-9415fcdf3f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:00.702390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c634e44-2cc5-4214-b300-9415fcdf3f36 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.702680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c634e44-2cc5-4214-b300-9415fcdf3f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.707386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c634e44-2cc5-4214-b300-9415fcdf3f36 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.707526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 469/963] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:00 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 22 11:02:00.707622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c634e44-2cc5-4214-b300-9415fcdf3f36 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:00.707708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c60ad-fabc-449c-8888-e279fdd1bc0d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:00.707813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c634e44-2cc5-4214-b300-9415fcdf3f36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:00.707813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c634e44-2cc5-4214-b300-9415fcdf3f36 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:00.713172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 495/964] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:02:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:00.717862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c634e44-2cc5-4214-b300-9415fcdf3f36 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:00.736355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:00.807860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:00.820244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c60ad-fabc-449c-8888-e279fdd1bc0d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:00.827243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c634e44-2cc5-4214-b300-9415fcdf3f36 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:00.856153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae5330-0857-4478-9920-dbe69d862dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:00.856937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae5330-0857-4478-9920-dbe69d862dfb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.857232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae5330-0857-4478-9920-dbe69d862dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.857514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae5330-0857-4478-9920-dbe69d862dfb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.858166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae5330-0857-4478-9920-dbe69d862dfb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:00.859570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae5330-0857-4478-9920-dbe69d862dfb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:00.864978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bc5f1-92cc-4cbc-82d3-41f39ce93234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:00.865779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bc5f1-92cc-4cbc-82d3-41f39ce93234 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:00.866067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bc5f1-92cc-4cbc-82d3-41f39ce93234 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:00.866343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bc5f1-92cc-4cbc-82d3-41f39ce93234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-83ae5330-0857-4478-9920-dbe69d862dfb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:00.872823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:00.873857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:00.880430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 470/965] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:02:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:00.881255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae5330-0857-4478-9920-dbe69d862dfb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:00.883329 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bc5f1-92cc-4cbc-82d3-41f39ce93234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:00.884800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bc5f1-92cc-4cbc-82d3-41f39ce93234 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1e3bc5f1-92cc-4cbc-82d3-41f39ce93234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:02:00.902189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:00.907942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:00.909071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:02:00.909071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:02:00.909071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:00.909071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:00.909071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 496/966] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:00.909071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bc5f1-92cc-4cbc-82d3-41f39ce93234 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:00.984123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:00.998397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae5330-0857-4478-9920-dbe69d862dfb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:01.014126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bc5f1-92cc-4cbc-82d3-41f39ce93234 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:01.033511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345811c-e1cf-4db4-8047-91c33e6a1a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:01.034182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345811c-e1cf-4db4-8047-91c33e6a1a02 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:01.034472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345811c-e1cf-4db4-8047-91c33e6a1a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:01.034759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345811c-e1cf-4db4-8047-91c33e6a1a02 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:01.035498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345811c-e1cf-4db4-8047-91c33e6a1a02 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:01.035811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345811c-e1cf-4db4-8047-91c33e6a1a02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:01.037436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345811c-e1cf-4db4-8047-91c33e6a1a02 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:01.047289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 471/967] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:02:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:01.047289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345811c-e1cf-4db4-8047-91c33e6a1a02 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:01.062004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8707f-a613-4430-8ae2-1b1ef41af8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:01.062855 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8707f-a613-4430-8ae2-1b1ef41af8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:01.063286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8707f-a613-4430-8ae2-1b1ef41af8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:01.063670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8707f-a613-4430-8ae2-1b1ef41af8f0 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:01.064680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8707f-a613-4430-8ae2-1b1ef41af8f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:01.064998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8707f-a613-4430-8ae2-1b1ef41af8f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:01.066856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8707f-a613-4430-8ae2-1b1ef41af8f0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:01.069740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:01.074984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 497/968] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:02:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:01.081954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8707f-a613-4430-8ae2-1b1ef41af8f0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:01.173983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345811c-e1cf-4db4-8047-91c33e6a1a02 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:01.215592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdf258-cd6d-4971-9dfb-63a12fd6baf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:01.216278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdf258-cd6d-4971-9dfb-63a12fd6baf2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:01.216569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdf258-cd6d-4971-9dfb-63a12fd6baf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:01.216848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdf258-cd6d-4971-9dfb-63a12fd6baf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:01.217586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdf258-cd6d-4971-9dfb-63a12fd6baf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:01.217896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdf258-cd6d-4971-9dfb-63a12fd6baf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:01.219422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdf258-cd6d-4971-9dfb-63a12fd6baf2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:01.232660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8707f-a613-4430-8ae2-1b1ef41af8f0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:01.236594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 472/969] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:01.245681 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdf258-cd6d-4971-9dfb-63a12fd6baf2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:01.261879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:01.281577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6c3f7-4719-4ed9-ad7a-1364de6575fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:01.282395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6c3f7-4719-4ed9-ad7a-1364de6575fa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:01.282808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6c3f7-4719-4ed9-ad7a-1364de6575fa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:01.283194 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6c3f7-4719-4ed9-ad7a-1364de6575fa None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:01.284303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6c3f7-4719-4ed9-ad7a-1364de6575fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:01.284718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6c3f7-4719-4ed9-ad7a-1364de6575fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:01.287124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6c3f7-4719-4ed9-ad7a-1364de6575fa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:01.295315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 498/970] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:02:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:01.300271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6c3f7-4719-4ed9-ad7a-1364de6575fa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:01.375634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:01.378086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdf258-cd6d-4971-9dfb-63a12fd6baf2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:01.417060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73faaa-ace1-4667-b8b9-eda1f6ba6210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:01.417504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6c3f7-4719-4ed9-ad7a-1364de6575fa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:01.419053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73faaa-ace1-4667-b8b9-eda1f6ba6210 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:01.419882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73faaa-ace1-4667-b8b9-eda1f6ba6210 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:01.420644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73faaa-ace1-4667-b8b9-eda1f6ba6210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:01.430579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73faaa-ace1-4667-b8b9-eda1f6ba6210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:01.432471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73faaa-ace1-4667-b8b9-eda1f6ba6210 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ef73faaa-ace1-4667-b8b9-eda1f6ba6210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:02:01.447593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:01.448518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:01.449474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:02:01.449474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:02:01.449474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:02:01.449474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:02:01.449474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:02:01.449474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:02:01.449474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:02:01.449474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:01.456744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3980c-e80d-48b7-9e54-d63e9c835f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:01.456744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3980c-e80d-48b7-9e54-d63e9c835f12 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:01.456744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3980c-e80d-48b7-9e54-d63e9c835f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:01.456744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3980c-e80d-48b7-9e54-d63e9c835f12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:01.457274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 473/971] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:01.457579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3980c-e80d-48b7-9e54-d63e9c835f12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:01.458586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73faaa-ace1-4667-b8b9-eda1f6ba6210 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:01.458942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3980c-e80d-48b7-9e54-d63e9c835f12 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:01.491282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 499/972] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 22 11:02:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:02:01.552448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:01.587958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73faaa-ace1-4667-b8b9-eda1f6ba6210 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:01.640239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ac8ac-9103-4f2e-a117-986d922c743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:01.641240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ac8ac-9103-4f2e-a117-986d922c743b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:01.641709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ac8ac-9103-4f2e-a117-986d922c743b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:01.642186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ac8ac-9103-4f2e-a117-986d922c743b None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:01.643166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ac8ac-9103-4f2e-a117-986d922c743b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:01.643687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ac8ac-9103-4f2e-a117-986d922c743b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:01.645327 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ac8ac-9103-4f2e-a117-986d922c743b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:01.659717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 474/973] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:02:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:01.666065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3980c-e80d-48b7-9e54-d63e9c835f12 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:01.777631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3980c-e80d-48b7-9e54-d63e9c835f12 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:01.799582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:01.816283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d878df-420a-4d0a-a6b4-4f66fb8f781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:01.817347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d878df-420a-4d0a-a6b4-4f66fb8f781e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:01.817844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d878df-420a-4d0a-a6b4-4f66fb8f781e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:01.818339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d878df-420a-4d0a-a6b4-4f66fb8f781e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:01.819433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d878df-420a-4d0a-a6b4-4f66fb8f781e None admin] RBAC: Adding query filter params (role.id=19b08ccf876549a4b4f59fd06c217412, scope.project.id=89224d740c7148938c2d803804083a57, user.id=646d7c5e6c8e4825a7fe06d018ee319c) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:01.819984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d878df-420a-4d0a-a6b4-4f66fb8f781e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:01.821662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d878df-420a-4d0a-a6b4-4f66fb8f781e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:01.834571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 500/974] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:02:01 2022] GET /identity/v3/role_assignments?user.id=646d7c5e6c8e4825a7fe06d018ee319c&scope.project.id=89224d740c7148938c2d803804083a57&role.id=19b08ccf876549a4b4f59fd06c217412 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:02.258357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af097330-e2b4-4397-b7d9-f6ab345a4bf2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:02.258906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af097330-e2b4-4397-b7d9-f6ab345a4bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:02.258906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af097330-e2b4-4397-b7d9-f6ab345a4bf2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:02.260558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 475/975] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:02.265945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8c492-e2cc-439c-b4a7-00bf5f98769c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:02.266339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8c492-e2cc-439c-b4a7-00bf5f98769c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:02.266526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8c492-e2cc-439c-b4a7-00bf5f98769c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:02.284665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5cd8c492-e2cc-439c-b4a7-00bf5f98769c None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:02.285851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5cd8c492-e2cc-439c-b4a7-00bf5f98769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:02.353662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 501/976] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:02.378183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af097330-e2b4-4397-b7d9-f6ab345a4bf2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:02.481966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af097330-e2b4-4397-b7d9-f6ab345a4bf2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:02.509205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:02.517359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:02.518037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:02.518325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:02.518612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:02.519519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:02.520978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:02.522288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WARNING keystone.server.flask.common [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 22 11:02:02.535579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.sql.core [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 22 11:02:02.535579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 22 11:02:02.535579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [parameters: {'user_id': '0de4c3d9a70c4761a81d89b0966bef34', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 22 11:02:02.535579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80394) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 22 11:02:02.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 22 11:02:02.553096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application yield resource Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.rollback() Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.force_reraise() Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise self.value Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application session.commit() Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 22 11:02:02.554253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._prepare_impl() Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.flush() Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._flush(objects) Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application compat.raise_( Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application flush_context.execute() Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rec.execute(self) Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 22 11:02:02.556525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 22 11:02:02.558598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 22 11:02:02.560951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [parameters: {'user_id': '0de4c3d9a70c4761a81d89b0966bef34', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:02.564490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 22 11:02:02.566313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 22 11:02:02.568488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 22 11:02:02.568488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 22 11:02:02.568488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 22 11:02:02.568488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 22 11:02:02.568488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 22 11:02:02.568488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:02.568488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 476/977] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Jul 22 11:02:02 2022] POST /identity/v3/users => generated 160 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:02.569378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8c492-e2cc-439c-b4a7-00bf5f98769c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:02.669728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8c492-e2cc-439c-b4a7-00bf5f98769c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:02.686392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:02.707844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aff65f-2aaa-4e6c-9128-784627900810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:02.708641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aff65f-2aaa-4e6c-9128-784627900810 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:02.708933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aff65f-2aaa-4e6c-9128-784627900810 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:02.709213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aff65f-2aaa-4e6c-9128-784627900810 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:02.716661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aff65f-2aaa-4e6c-9128-784627900810 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:02.718126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aff65f-2aaa-4e6c-9128-784627900810 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d2aff65f-2aaa-4e6c-9128-784627900810 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:02:02.733057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:02.735143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:02.737351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 502/978] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:02:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:02.744734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:02.848354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c08b83-867c-468f-9ebd-5064b0f51b13 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:02.883500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701115d-b143-427f-bda4-781e92bc47cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:02.884195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701115d-b143-427f-bda4-781e92bc47cb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:02.884485 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701115d-b143-427f-bda4-781e92bc47cb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:02.884819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701115d-b143-427f-bda4-781e92bc47cb None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:02.885699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701115d-b143-427f-bda4-781e92bc47cb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:02.886009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701115d-b143-427f-bda4-781e92bc47cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:02.887745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701115d-b143-427f-bda4-781e92bc47cb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:02.926474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 477/979] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Jul 22 11:02:02 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:03.213889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d17ca-ba54-4499-b9b0-f6c1f6afaae9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:03.214207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d17ca-ba54-4499-b9b0-f6c1f6afaae9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:03.214484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d17ca-ba54-4499-b9b0-f6c1f6afaae9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:03.216061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 503/980] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:03.221281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965751a-aa5f-4188-9269-451a7861e107 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:03.221589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965751a-aa5f-4188-9269-451a7861e107 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:03.221879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965751a-aa5f-4188-9269-451a7861e107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:03.239614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3965751a-aa5f-4188-9269-451a7861e107 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:03.241145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3965751a-aa5f-4188-9269-451a7861e107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:03.286408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:03.311847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 478/981] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:03.334347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d17ca-ba54-4499-b9b0-f6c1f6afaae9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:03.436194 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d17ca-ba54-4499-b9b0-f6c1f6afaae9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:03.469746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457df06a-5458-4f1b-8348-25a9fc37acd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:03.470344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457df06a-5458-4f1b-8348-25a9fc37acd8 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:03.470634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457df06a-5458-4f1b-8348-25a9fc37acd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:03.470916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457df06a-5458-4f1b-8348-25a9fc37acd8 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:03.471684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457df06a-5458-4f1b-8348-25a9fc37acd8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:03.473066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457df06a-5458-4f1b-8348-25a9fc37acd8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:03.495296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 504/982] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:02:03 2022] POST /identity/v3/services => generated 242 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:02:03.656301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082c56b-7c81-4aa6-9f1a-c77ba3c964a7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:03.656592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082c56b-7c81-4aa6-9f1a-c77ba3c964a7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:03.656871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082c56b-7c81-4aa6-9f1a-c77ba3c964a7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:03.658651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 479/983] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:03.663581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1398e-74d7-4044-92e5-57c68d98591f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:03.663882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1398e-74d7-4044-92e5-57c68d98591f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:03.664214 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1398e-74d7-4044-92e5-57c68d98591f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:03.681974 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-95a1398e-74d7-4044-92e5-57c68d98591f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:03.683173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-95a1398e-74d7-4044-92e5-57c68d98591f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:03.710718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:03.761643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 505/984] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:03.783941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082c56b-7c81-4aa6-9f1a-c77ba3c964a7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:03.887302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082c56b-7c81-4aa6-9f1a-c77ba3c964a7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:03.921489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c84f28-f4c9-498e-aed6-cfe8f50a1a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:03.922300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c84f28-f4c9-498e-aed6-cfe8f50a1a2b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:03.922585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c84f28-f4c9-498e-aed6-cfe8f50a1a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:03.922861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c84f28-f4c9-498e-aed6-cfe8f50a1a2b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:03.923539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c84f28-f4c9-498e-aed6-cfe8f50a1a2b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:03.924908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c84f28-f4c9-498e-aed6-cfe8f50a1a2b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a3c84f28-f4c9-498e-aed6-cfe8f50a1a2b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:03.937667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:03.938660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:02:03.939964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:03.943349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 480/985] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:02:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:03.944960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1398e-74d7-4044-92e5-57c68d98591f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:03.969850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f81a6d-3360-44f0-b758-86d482476cec None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:03.970274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f81a6d-3360-44f0-b758-86d482476cec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:03.970473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f81a6d-3360-44f0-b758-86d482476cec None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:03.972003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 481/986] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:03.979641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccd147-17be-4376-b464-d50ba5905509 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:03.979992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccd147-17be-4376-b464-d50ba5905509 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:03.980203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccd147-17be-4376-b464-d50ba5905509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:03.998732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-35ccd147-17be-4376-b464-d50ba5905509 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:03.999946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-35ccd147-17be-4376-b464-d50ba5905509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:04.039791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:04.052119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1398e-74d7-4044-92e5-57c68d98591f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:04.083432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 482/987] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:02:04.094476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81a885-d6b6-448e-9ae4-38a8ff57532b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:04.095142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81a885-d6b6-448e-9ae4-38a8ff57532b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:04.095466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81a885-d6b6-448e-9ae4-38a8ff57532b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:04.095744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81a885-d6b6-448e-9ae4-38a8ff57532b None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:04.096446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81a885-d6b6-448e-9ae4-38a8ff57532b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:04.096751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81a885-d6b6-448e-9ae4-38a8ff57532b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:04.098113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81a885-d6b6-448e-9ae4-38a8ff57532b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:04.103118 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccd147-17be-4376-b464-d50ba5905509 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:04.108196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 506/988] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:02:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:04.109513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81a885-d6b6-448e-9ae4-38a8ff57532b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:04.135434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:04.217029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81a885-d6b6-448e-9ae4-38a8ff57532b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:04.224043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccd147-17be-4376-b464-d50ba5905509 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:04.253928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30d7ac-2e05-40a0-9865-6a677eb3f9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:04.254912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30d7ac-2e05-40a0-9865-6a677eb3f9ae None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:04.255461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30d7ac-2e05-40a0-9865-6a677eb3f9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:04.255941 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30d7ac-2e05-40a0-9865-6a677eb3f9ae None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:04.257490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30d7ac-2e05-40a0-9865-6a677eb3f9ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:04.257900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30d7ac-2e05-40a0-9865-6a677eb3f9ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:04.259431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30d7ac-2e05-40a0-9865-6a677eb3f9ae None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:04.271871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 507/989] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:02:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:04.272099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacd969-cccb-4ae3-9663-6ceb0afac41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:04.272099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacd969-cccb-4ae3-9663-6ceb0afac41a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:04.272099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacd969-cccb-4ae3-9663-6ceb0afac41a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:04.272099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacd969-cccb-4ae3-9663-6ceb0afac41a None admin] PATH_INFO: `/v3/services/image` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:04.272529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacd969-cccb-4ae3-9663-6ceb0afac41a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:04.272529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacd969-cccb-4ae3-9663-6ceb0afac41a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:04.274302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30d7ac-2e05-40a0-9865-6a677eb3f9ae None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4dacd969-cccb-4ae3-9663-6ceb0afac41a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:04.283774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:04.284944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:04.285795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 483/990] 10.222.0.20 () {56 vars in 1105 bytes} [Fri Jul 22 11:02:04 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:04.291209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacd969-cccb-4ae3-9663-6ceb0afac41a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:04.327489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:04.369942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:04.382387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30d7ac-2e05-40a0-9865-6a677eb3f9ae None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:04.395384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacd969-cccb-4ae3-9663-6ceb0afac41a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:04.421692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14a914-98a8-4227-bca8-83e6027e1872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:04.422353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14a914-98a8-4227-bca8-83e6027e1872 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:04.422640 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14a914-98a8-4227-bca8-83e6027e1872 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:04.422969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14a914-98a8-4227-bca8-83e6027e1872 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:04.423844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14a914-98a8-4227-bca8-83e6027e1872 None admin] RBAC: Adding query filter params (interface=admin, service_id=50211284bb164417913688903755edf7, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:04.424172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14a914-98a8-4227-bca8-83e6027e1872 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:04.425539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14a914-98a8-4227-bca8-83e6027e1872 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:04.431207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 508/991] 10.222.0.20 () {56 vars in 1249 bytes} [Fri Jul 22 11:02:04 2022] GET /identity/v3/endpoints?service_id=50211284bb164417913688903755edf7&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:04.432106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebc9d3-1e70-46ec-ab0f-5cfc76d23361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:04.432775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebc9d3-1e70-46ec-ab0f-5cfc76d23361 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:04.433064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebc9d3-1e70-46ec-ab0f-5cfc76d23361 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:04.433345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebc9d3-1e70-46ec-ab0f-5cfc76d23361 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:04.434051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebc9d3-1e70-46ec-ab0f-5cfc76d23361 None admin] RBAC: Adding query filter params (name=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:04.434362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebc9d3-1e70-46ec-ab0f-5cfc76d23361 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:04.435769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebc9d3-1e70-46ec-ab0f-5cfc76d23361 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:04.436517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14a914-98a8-4227-bca8-83e6027e1872 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:04.446150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 484/992] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:04.447543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebc9d3-1e70-46ec-ab0f-5cfc76d23361 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:04.541698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14a914-98a8-4227-bca8-83e6027e1872 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:04.549888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:04.553441 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebc9d3-1e70-46ec-ab0f-5cfc76d23361 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:04.580407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadaf3a-416b-46eb-a82a-04b7b364ee6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:04.581089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadaf3a-416b-46eb-a82a-04b7b364ee6f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:04.581499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadaf3a-416b-46eb-a82a-04b7b364ee6f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:04.581787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadaf3a-416b-46eb-a82a-04b7b364ee6f None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:04.582471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbadaf3a-416b-46eb-a82a-04b7b364ee6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:04.583976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbadaf3a-416b-46eb-a82a-04b7b364ee6f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:04.589875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5df4ad-0c48-4d6d-af9d-5abc43811416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:04.590543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5df4ad-0c48-4d6d-af9d-5abc43811416 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:04.590828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5df4ad-0c48-4d6d-af9d-5abc43811416 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:04.591290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5df4ad-0c48-4d6d-af9d-5abc43811416 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:04.595329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5df4ad-0c48-4d6d-af9d-5abc43811416 None admin] RBAC: Adding query filter params (type=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:04.595329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5df4ad-0c48-4d6d-af9d-5abc43811416 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:04.595329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5df4ad-0c48-4d6d-af9d-5abc43811416 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:04.599719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 509/993] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 22 11:02:04 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:02:04.615601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 485/994] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:05.544591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9c1df-bb8c-477e-9206-44a300cf1627 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:05.545143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9c1df-bb8c-477e-9206-44a300cf1627 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:05.545413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9c1df-bb8c-477e-9206-44a300cf1627 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:05.546847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 510/995] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:05.551727 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2af2c5-9cea-461c-b1b4-44c4e3fafb1a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:05.552124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2af2c5-9cea-461c-b1b4-44c4e3fafb1a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:05.552284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2af2c5-9cea-461c-b1b4-44c4e3fafb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:05.569546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dc2af2c5-9cea-461c-b1b4-44c4e3fafb1a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:05.570757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dc2af2c5-9cea-461c-b1b4-44c4e3fafb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:05.645598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 486/996] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:05.668291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9c1df-bb8c-477e-9206-44a300cf1627 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:05.687513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:05.778652 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9c1df-bb8c-477e-9206-44a300cf1627 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:05.812331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3696b53-5865-43d8-a318-460283274989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:05.813051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3696b53-5865-43d8-a318-460283274989 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:05.813339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3696b53-5865-43d8-a318-460283274989 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:05.813620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3696b53-5865-43d8-a318-460283274989 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:05.814263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3696b53-5865-43d8-a318-460283274989 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:05.815651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3696b53-5865-43d8-a318-460283274989 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b3696b53-5865-43d8-a318-460283274989 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:05.827081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:05.828321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:05.829203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 511/997] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:02:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:05.836535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2af2c5-9cea-461c-b1b4-44c4e3fafb1a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:05.927064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:05.935043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2af2c5-9cea-461c-b1b4-44c4e3fafb1a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:05.968174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc59d7-7c4e-4208-8fd0-683165c80a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:05.968831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc59d7-7c4e-4208-8fd0-683165c80a5e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:05.969115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc59d7-7c4e-4208-8fd0-683165c80a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:05.969392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc59d7-7c4e-4208-8fd0-683165c80a5e None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:05.970088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc59d7-7c4e-4208-8fd0-683165c80a5e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:05.970393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc59d7-7c4e-4208-8fd0-683165c80a5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:05.971797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc59d7-7c4e-4208-8fd0-683165c80a5e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:05.981694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 487/998] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:02:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:05.982948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3696b53-5865-43d8-a318-460283274989 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:06.065087 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:06.085740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3696b53-5865-43d8-a318-460283274989 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:06.119195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443796c4-59c6-439a-945f-63b5804942cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:06.119889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443796c4-59c6-439a-945f-63b5804942cb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:06.120178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443796c4-59c6-439a-945f-63b5804942cb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:06.120458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443796c4-59c6-439a-945f-63b5804942cb None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:06.121161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443796c4-59c6-439a-945f-63b5804942cb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:06.121472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443796c4-59c6-439a-945f-63b5804942cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:06.122836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443796c4-59c6-439a-945f-63b5804942cb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:06.132624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 512/999] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:02:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:06.134615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc59d7-7c4e-4208-8fd0-683165c80a5e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:06.239607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc59d7-7c4e-4208-8fd0-683165c80a5e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:06.274421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82f864-83b6-4606-a594-80791d800b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:06.275171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82f864-83b6-4606-a594-80791d800b96 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:06.275493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82f864-83b6-4606-a594-80791d800b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:06.275781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82f864-83b6-4606-a594-80791d800b96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:06.276578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82f864-83b6-4606-a594-80791d800b96 None admin] RBAC: Adding query filter params (interface=public, service_id=85d68629981c4c11a34b8b0699fb1010, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:06.276898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82f864-83b6-4606-a594-80791d800b96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:06.278271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82f864-83b6-4606-a594-80791d800b96 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:06.287876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 488/1000] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 22 11:02:06 2022] GET /identity/v3/endpoints?service_id=85d68629981c4c11a34b8b0699fb1010&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:06.289060 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443796c4-59c6-439a-945f-63b5804942cb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:06.390108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443796c4-59c6-439a-945f-63b5804942cb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:06.423453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08943e7c-17dd-4849-9b2a-c9bc7057a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:06.424122 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08943e7c-17dd-4849-9b2a-c9bc7057a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:06.424413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08943e7c-17dd-4849-9b2a-c9bc7057a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:06.424693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08943e7c-17dd-4849-9b2a-c9bc7057a4f4 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:06.425369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08943e7c-17dd-4849-9b2a-c9bc7057a4f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:06.426748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08943e7c-17dd-4849-9b2a-c9bc7057a4f4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:06.439348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 513/1001] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 22 11:02:06 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:02:06.626454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c2e1-35f3-45c2-942d-f045b016743c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:06.626990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c2e1-35f3-45c2-942d-f045b016743c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:06.626990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c2e1-35f3-45c2-942d-f045b016743c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:06.628577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 489/1002] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:06.633614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9ccff-cd6f-46ff-9d1b-4b3bf67969a1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:06.634082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9ccff-cd6f-46ff-9d1b-4b3bf67969a1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:06.634160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9ccff-cd6f-46ff-9d1b-4b3bf67969a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:06.651553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-61e9ccff-cd6f-46ff-9d1b-4b3bf67969a1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:06.652671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-61e9ccff-cd6f-46ff-9d1b-4b3bf67969a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:06.654711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:06.655359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b479d73-0946-4b27-8483-0821a573f85b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:06.655653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b479d73-0946-4b27-8483-0821a573f85b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:06.655965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b479d73-0946-4b27-8483-0821a573f85b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:06.657616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 490/1003] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:06.662550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c848f2-3cf1-4281-96d2-d762da6ea384 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:06.662845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c848f2-3cf1-4281-96d2-d762da6ea384 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:06.663124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c848f2-3cf1-4281-96d2-d762da6ea384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:06.679857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-47c848f2-3cf1-4281-96d2-d762da6ea384 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:06.681042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-47c848f2-3cf1-4281-96d2-d762da6ea384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:06.697957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:06.730312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 514/1004] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:06.751859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 491/1005] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:02:06.753040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9ccff-cd6f-46ff-9d1b-4b3bf67969a1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:06.775832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c848f2-3cf1-4281-96d2-d762da6ea384 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:06.861174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9ccff-cd6f-46ff-9d1b-4b3bf67969a1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:06.884108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c848f2-3cf1-4281-96d2-d762da6ea384 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:06.899567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727563d3-5fc4-4189-91ce-56671f73ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:06.900388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727563d3-5fc4-4189-91ce-56671f73ecff None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:06.900671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727563d3-5fc4-4189-91ce-56671f73ecff None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:06.900947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727563d3-5fc4-4189-91ce-56671f73ecff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:06.901662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727563d3-5fc4-4189-91ce-56671f73ecff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:06.902962 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727563d3-5fc4-4189-91ce-56671f73ecff None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-727563d3-5fc4-4189-91ce-56671f73ecff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:06.915335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:06.916369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:06.917235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 515/1006] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Jul 22 11:02:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:06.922334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727563d3-5fc4-4189-91ce-56671f73ecff None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:06.924139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0de50-33a6-4136-b5bf-4d0e85622eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:06.924848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0de50-33a6-4136-b5bf-4d0e85622eeb None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:06.924953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:06.925142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0de50-33a6-4136-b5bf-4d0e85622eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:06.925426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0de50-33a6-4136-b5bf-4d0e85622eeb None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:06.926186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0de50-33a6-4136-b5bf-4d0e85622eeb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:06.927673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0de50-33a6-4136-b5bf-4d0e85622eeb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:06.947016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 492/1007] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:02:06 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:02:07.035446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727563d3-5fc4-4189-91ce-56671f73ecff None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:07.077782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53c90c-f695-4234-8186-30623e751d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:07.078461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53c90c-f695-4234-8186-30623e751d7a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:07.078745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53c90c-f695-4234-8186-30623e751d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:07.079024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53c90c-f695-4234-8186-30623e751d7a None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:07.079787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53c90c-f695-4234-8186-30623e751d7a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:07.080096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53c90c-f695-4234-8186-30623e751d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:07.081472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53c90c-f695-4234-8186-30623e751d7a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:07.092120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 516/1008] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:02:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:07.093392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0de50-33a6-4136-b5bf-4d0e85622eeb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:07.110943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:07.201611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0de50-33a6-4136-b5bf-4d0e85622eeb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:07.234174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a02084-343e-4d6a-bde0-e9b09ae35f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:07.234849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a02084-343e-4d6a-bde0-e9b09ae35f6a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:07.235138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a02084-343e-4d6a-bde0-e9b09ae35f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:07.235449 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a02084-343e-4d6a-bde0-e9b09ae35f6a None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:07.236158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a02084-343e-4d6a-bde0-e9b09ae35f6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:07.236470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a02084-343e-4d6a-bde0-e9b09ae35f6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:07.237842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a02084-343e-4d6a-bde0-e9b09ae35f6a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:07.243760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 493/1009] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Jul 22 11:02:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:07.249262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53c90c-f695-4234-8186-30623e751d7a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:07.310666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:07.350903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53c90c-f695-4234-8186-30623e751d7a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:07.385050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f633fa-3400-46a3-b70b-33a3bbf5b050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:07.385712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f633fa-3400-46a3-b70b-33a3bbf5b050 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:07.386004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f633fa-3400-46a3-b70b-33a3bbf5b050 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:07.386284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f633fa-3400-46a3-b70b-33a3bbf5b050 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:07.387043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f633fa-3400-46a3-b70b-33a3bbf5b050 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:07.388445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f633fa-3400-46a3-b70b-33a3bbf5b050 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:07.403496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 517/1010] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 22 11:02:07 2022] POST /identity/v3/endpoints => generated 325 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:02:08.488861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53da16-280c-491f-bc02-b18810322654 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:08.489372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53da16-280c-491f-bc02-b18810322654 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:08.489451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53da16-280c-491f-bc02-b18810322654 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:08.491299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 494/1011] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:08.496185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6803e-38b8-4a0a-8286-22e2c211c856 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:08.496591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6803e-38b8-4a0a-8286-22e2c211c856 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:08.496775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6803e-38b8-4a0a-8286-22e2c211c856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:08.514150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7eb6803e-38b8-4a0a-8286-22e2c211c856 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:08.515381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7eb6803e-38b8-4a0a-8286-22e2c211c856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:08.597984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 518/1012] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:08 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:08.620315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53da16-280c-491f-bc02-b18810322654 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:08.702904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:08.729104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53da16-280c-491f-bc02-b18810322654 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:08.763425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080abcf-c91a-4805-ace7-77619db0b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:08.764153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080abcf-c91a-4805-ace7-77619db0b29c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:08.764443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080abcf-c91a-4805-ace7-77619db0b29c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:08.764724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080abcf-c91a-4805-ace7-77619db0b29c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:08.765372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080abcf-c91a-4805-ace7-77619db0b29c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:08.766742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080abcf-c91a-4805-ace7-77619db0b29c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2080abcf-c91a-4805-ace7-77619db0b29c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:08.779788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:08.780785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:08.786531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 495/1013] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:02:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:08.789202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6803e-38b8-4a0a-8286-22e2c211c856 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:08.838696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:08.891812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6803e-38b8-4a0a-8286-22e2c211c856 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:08.925152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779214-d3b6-46c5-87f9-c7c87229c4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:08.925827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779214-d3b6-46c5-87f9-c7c87229c4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:08.926111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779214-d3b6-46c5-87f9-c7c87229c4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:08.926386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779214-d3b6-46c5-87f9-c7c87229c4f5 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:08.927085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9779214-d3b6-46c5-87f9-c7c87229c4f5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:08.927417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9779214-d3b6-46c5-87f9-c7c87229c4f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:08.928789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9779214-d3b6-46c5-87f9-c7c87229c4f5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:08.935416 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 519/1014] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:02:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:08.940414 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080abcf-c91a-4805-ace7-77619db0b29c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:09.026366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b2986-45ae-4c6d-89e8-c412f4d641a9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.026606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b2986-45ae-4c6d-89e8-c412f4d641a9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.026863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b2986-45ae-4c6d-89e8-c412f4d641a9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.028565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 520/1015] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:02:09.033882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545b4c2-8f4e-4479-b5e6-0f5d62987fa5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.034175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545b4c2-8f4e-4479-b5e6-0f5d62987fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.034460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545b4c2-8f4e-4479-b5e6-0f5d62987fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.052219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2545b4c2-8f4e-4479-b5e6-0f5d62987fa5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:09.052766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080abcf-c91a-4805-ace7-77619db0b29c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:09.053400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2545b4c2-8f4e-4479-b5e6-0f5d62987fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:09.089406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bd273-0d4e-4c81-b4a1-09a5a491442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:09.090370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bd273-0d4e-4c81-b4a1-09a5a491442a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.090833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bd273-0d4e-4c81-b4a1-09a5a491442a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.091341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bd273-0d4e-4c81-b4a1-09a5a491442a None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.092365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bd273-0d4e-4c81-b4a1-09a5a491442a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:09.092907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bd273-0d4e-4c81-b4a1-09a5a491442a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:09.094639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bd273-0d4e-4c81-b4a1-09a5a491442a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:09.105719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 496/1016] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Jul 22 11:02:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:09.107690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bd273-0d4e-4c81-b4a1-09a5a491442a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:09.113921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:09.147644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 521/1017] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:09 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:09.163708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545b4c2-8f4e-4479-b5e6-0f5d62987fa5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:09.211354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:09.263041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bd273-0d4e-4c81-b4a1-09a5a491442a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:09.283872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545b4c2-8f4e-4479-b5e6-0f5d62987fa5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:09.308062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848023c-18d8-47c2-a872-eafa7e71b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:09.308743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848023c-18d8-47c2-a872-eafa7e71b95b None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.309030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848023c-18d8-47c2-a872-eafa7e71b95b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.309408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848023c-18d8-47c2-a872-eafa7e71b95b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.310046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848023c-18d8-47c2-a872-eafa7e71b95b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:09.311457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848023c-18d8-47c2-a872-eafa7e71b95b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:09.322976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af15c4e-bd19-4241-a8b0-d97512931339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:09.323915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af15c4e-bd19-4241-a8b0-d97512931339 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.324245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af15c4e-bd19-4241-a8b0-d97512931339 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.324575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af15c4e-bd19-4241-a8b0-d97512931339 None admin] PATH_INFO: `/v3/services/image` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.325332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af15c4e-bd19-4241-a8b0-d97512931339 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:09.326865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af15c4e-bd19-4241-a8b0-d97512931339 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5af15c4e-bd19-4241-a8b0-d97512931339 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:09.339452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:09.340507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 22 11:02:09.341384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:09.342971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 497/1018] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 22 11:02:09 2022] POST /identity/v3/endpoints => generated 334 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:02:09.346457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 522/1019] 10.222.0.20 () {56 vars in 1105 bytes} [Fri Jul 22 11:02:09 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:09.347568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848023c-18d8-47c2-a872-eafa7e71b95b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:09.451125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:09.469013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a914-191a-499b-8b17-2eecd3865f53 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.469360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a914-191a-499b-8b17-2eecd3865f53 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.469596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a914-191a-499b-8b17-2eecd3865f53 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.471169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 523/1020] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:02:09.477165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5362f3-c151-46c6-8eb0-c1181d8fe31a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.477464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5362f3-c151-46c6-8eb0-c1181d8fe31a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.477923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5362f3-c151-46c6-8eb0-c1181d8fe31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.497043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9c5362f3-c151-46c6-8eb0-c1181d8fe31a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:09.498285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c5362f3-c151-46c6-8eb0-c1181d8fe31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:09.506819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848023c-18d8-47c2-a872-eafa7e71b95b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:09.547351 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f43263-a510-4569-8b57-4fe231f41e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:09.548077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f43263-a510-4569-8b57-4fe231f41e91 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.548861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f43263-a510-4569-8b57-4fe231f41e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.548861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f43263-a510-4569-8b57-4fe231f41e91 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.550176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f43263-a510-4569-8b57-4fe231f41e91 None admin] RBAC: Adding query filter params (name=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:09.550176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f43263-a510-4569-8b57-4fe231f41e91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:09.551072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f43263-a510-4569-8b57-4fe231f41e91 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:09.556370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 498/1021] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:09.562442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f43263-a510-4569-8b57-4fe231f41e91 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:09.578263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 524/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:02:09.603145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5362f3-c151-46c6-8eb0-c1181d8fe31a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:09.651233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:09.677903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f43263-a510-4569-8b57-4fe231f41e91 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:09.700552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:09.717458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fab13b-d889-4ca9-9179-5f54026a5478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:09.718124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab13b-d889-4ca9-9179-5f54026a5478 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.718423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab13b-d889-4ca9-9179-5f54026a5478 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.718795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fab13b-d889-4ca9-9179-5f54026a5478 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.719549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab13b-d889-4ca9-9179-5f54026a5478 None admin] RBAC: Adding query filter params (type=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:09.719862 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab13b-d889-4ca9-9179-5f54026a5478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:09.721241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fab13b-d889-4ca9-9179-5f54026a5478 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:09.722163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5362f3-c151-46c6-8eb0-c1181d8fe31a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:09.732535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 499/1023] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:09.733556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fab13b-d889-4ca9-9179-5f54026a5478 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:09.766022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23506b7c-7b36-43da-8fd9-faa29ae553c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:09.767143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23506b7c-7b36-43da-8fd9-faa29ae553c8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.767463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23506b7c-7b36-43da-8fd9-faa29ae553c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.767800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23506b7c-7b36-43da-8fd9-faa29ae553c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.769183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23506b7c-7b36-43da-8fd9-faa29ae553c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:09.771562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23506b7c-7b36-43da-8fd9-faa29ae553c8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:09.778663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 525/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:09.779777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23506b7c-7b36-43da-8fd9-faa29ae553c8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:09.837547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fab13b-d889-4ca9-9179-5f54026a5478 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:09.873439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c65a28-f55d-46c0-93de-6d653d00c5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:09.874091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c65a28-f55d-46c0-93de-6d653d00c5ab None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.874668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c65a28-f55d-46c0-93de-6d653d00c5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.874668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c65a28-f55d-46c0-93de-6d653d00c5ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.875483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c65a28-f55d-46c0-93de-6d653d00c5ab None admin] RBAC: Adding query filter params (interface=public, service_id=c2cb148945f540fcb9eaba82c1fd534b, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:09.875763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c65a28-f55d-46c0-93de-6d653d00c5ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:09.877192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c65a28-f55d-46c0-93de-6d653d00c5ab None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:09.887250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 500/1025] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 22 11:02:09 2022] GET /identity/v3/endpoints?service_id=c2cb148945f540fcb9eaba82c1fd534b&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:09.888815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c65a28-f55d-46c0-93de-6d653d00c5ab None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:09.894156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23506b7c-7b36-43da-8fd9-faa29ae553c8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:09.897277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:09.934293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bd9d5-8aca-456f-8d7a-420dc89b6bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:09.935095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bd9d5-8aca-456f-8d7a-420dc89b6bca None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:09.935395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bd9d5-8aca-456f-8d7a-420dc89b6bca None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:09.935680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bd9d5-8aca-456f-8d7a-420dc89b6bca None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:09.936523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bd9d5-8aca-456f-8d7a-420dc89b6bca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:09.937954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bd9d5-8aca-456f-8d7a-420dc89b6bca None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:09.941919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:09.979293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 526/1026] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:02:09 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:02:10.014482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c65a28-f55d-46c0-93de-6d653d00c5ab None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:10.055362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b7c79-ac80-4055-bafc-32ba616ea0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:10.056103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b7c79-ac80-4055-bafc-32ba616ea0fc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:10.056395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b7c79-ac80-4055-bafc-32ba616ea0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:10.056675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b7c79-ac80-4055-bafc-32ba616ea0fc None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:10.057437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b7c79-ac80-4055-bafc-32ba616ea0fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:10.058910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b7c79-ac80-4055-bafc-32ba616ea0fc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:10.075544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 501/1027] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 22 11:02:09 2022] GET /identity/v3/services => generated 1957 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:02:11.510569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11373c5e-a7c7-42ff-9bcf-6429318abc72 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:11.511126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11373c5e-a7c7-42ff-9bcf-6429318abc72 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:11.511126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11373c5e-a7c7-42ff-9bcf-6429318abc72 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:11.512560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 527/1028] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:11.517733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f677855-4e99-4edf-9b7a-75253707a2d9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:11.518133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f677855-4e99-4edf-9b7a-75253707a2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:11.518331 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f677855-4e99-4edf-9b7a-75253707a2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:11.534994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1f677855-4e99-4edf-9b7a-75253707a2d9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:11.536209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1f677855-4e99-4edf-9b7a-75253707a2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:11.605007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 502/1029] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:11.628578 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11373c5e-a7c7-42ff-9bcf-6429318abc72 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:11.730748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11373c5e-a7c7-42ff-9bcf-6429318abc72 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:11.764525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243904-2f58-4cb4-868b-d188cc14d9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:11.765263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243904-2f58-4cb4-868b-d188cc14d9dc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:11.765553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243904-2f58-4cb4-868b-d188cc14d9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:11.765831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243904-2f58-4cb4-868b-d188cc14d9dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:11.776578 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97243904-2f58-4cb4-868b-d188cc14d9dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-97243904-2f58-4cb4-868b-d188cc14d9dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:11.784690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:11.785627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:02:11.786486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:11.791325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 528/1030] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 22 11:02:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:11.792811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f677855-4e99-4edf-9b7a-75253707a2d9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:11.837639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:11.895869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f677855-4e99-4edf-9b7a-75253707a2d9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:11.930723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00369ab1-d270-49c3-9c0a-6b1023422a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:11.931436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00369ab1-d270-49c3-9c0a-6b1023422a4d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:11.931726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00369ab1-d270-49c3-9c0a-6b1023422a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:11.932008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00369ab1-d270-49c3-9c0a-6b1023422a4d None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:11.932762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00369ab1-d270-49c3-9c0a-6b1023422a4d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:11.933072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00369ab1-d270-49c3-9c0a-6b1023422a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:11.934443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00369ab1-d270-49c3-9c0a-6b1023422a4d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:11.942609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 503/1031] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 22 11:02:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:11.948592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243904-2f58-4cb4-868b-d188cc14d9dc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:11.968384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:12.053217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243904-2f58-4cb4-868b-d188cc14d9dc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:12.087626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ee59f-4c07-432c-bbb3-042ba0f1c398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:12.088419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ee59f-4c07-432c-bbb3-042ba0f1c398 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.088703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ee59f-4c07-432c-bbb3-042ba0f1c398 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.088981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ee59f-4c07-432c-bbb3-042ba0f1c398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.099550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ee59f-4c07-432c-bbb3-042ba0f1c398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:12.100998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ee59f-4c07-432c-bbb3-042ba0f1c398 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:12.110994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004d055-0182-490d-84f5-5a5e77df857d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.111319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004d055-0182-490d-84f5-5a5e77df857d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.111606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004d055-0182-490d-84f5-5a5e77df857d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.113370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 504/1032] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:12.118012 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e624eef-6d18-4684-805c-5559734f8c70 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.118306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e624eef-6d18-4684-805c-5559734f8c70 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.118592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e624eef-6d18-4684-805c-5559734f8c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-617ee59f-4c07-432c-bbb3-042ba0f1c398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:02:12.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:12.119560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:12.120478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:02:12.120478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:02:12.120478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:12.120478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:12.123513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 529/1033] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:12.125554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ee59f-4c07-432c-bbb3-042ba0f1c398 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:12.138094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4e624eef-6d18-4684-805c-5559734f8c70 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:12.139757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4e624eef-6d18-4684-805c-5559734f8c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:12.196530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:12.207782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 505/1034] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:12.209182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a1202-5a8e-4bd2-bcb8-2679530a7fc9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.209544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a1202-5a8e-4bd2-bcb8-2679530a7fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.209825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a1202-5a8e-4bd2-bcb8-2679530a7fc9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.215428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 506/1035] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:12.219920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b3cb8-2682-4c52-aab7-2c21677a954e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.220460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b3cb8-2682-4c52-aab7-2c21677a954e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.220741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b3cb8-2682-4c52-aab7-2c21677a954e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.236507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ee59f-4c07-432c-bbb3-042ba0f1c398 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:12.240729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-334b3cb8-2682-4c52-aab7-2c21677a954e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:12.241938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-334b3cb8-2682-4c52-aab7-2c21677a954e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:12.268594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:12.271379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fb9bd-8ac5-4543-8fb5-d291c78ec7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:12.272051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fb9bd-8ac5-4543-8fb5-d291c78ec7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.272348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fb9bd-8ac5-4543-8fb5-d291c78ec7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.272628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fb9bd-8ac5-4543-8fb5-d291c78ec7d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.273928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fb9bd-8ac5-4543-8fb5-d291c78ec7d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:12.274292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fb9bd-8ac5-4543-8fb5-d291c78ec7d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:12.275760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fb9bd-8ac5-4543-8fb5-d291c78ec7d2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:12.287492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 530/1036] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:12.288288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fb9bd-8ac5-4543-8fb5-d291c78ec7d2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:12.310301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 507/1037] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:02:12.311079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b3cb8-2682-4c52-aab7-2c21677a954e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:12.387948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fb9bd-8ac5-4543-8fb5-d291c78ec7d2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:12.410369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b3cb8-2682-4c52-aab7-2c21677a954e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:12.413829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:12.422121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2618e-161a-418d-88bb-40a6743aebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:12.422909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2618e-161a-418d-88bb-40a6743aebbd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.423191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2618e-161a-418d-88bb-40a6743aebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.423497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2618e-161a-418d-88bb-40a6743aebbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.424670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2618e-161a-418d-88bb-40a6743aebbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:12.426093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2618e-161a-418d-88bb-40a6743aebbd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:12.427749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 531/1038] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:12.428593 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2618e-161a-418d-88bb-40a6743aebbd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:12.454498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f495de-3395-408f-a707-f35015ea83f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:12.455726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f495de-3395-408f-a707-f35015ea83f6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.456168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f495de-3395-408f-a707-f35015ea83f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.456552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f495de-3395-408f-a707-f35015ea83f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.466118 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f495de-3395-408f-a707-f35015ea83f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:12.467939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f495de-3395-408f-a707-f35015ea83f6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-47f495de-3395-408f-a707-f35015ea83f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:02:12.485494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:12.486569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 508/1039] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 22 11:02:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:12.488359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f495de-3395-408f-a707-f35015ea83f6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:12.513453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:12.534278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2618e-161a-418d-88bb-40a6743aebbd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:12.567286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f9095-8204-46df-b2be-8286954a5214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:12.568017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f9095-8204-46df-b2be-8286954a5214 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.568294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f9095-8204-46df-b2be-8286954a5214 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.568596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f9095-8204-46df-b2be-8286954a5214 None admin] PATH_INFO: `/v3/services/image` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.569244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f9095-8204-46df-b2be-8286954a5214 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:12.571173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f9095-8204-46df-b2be-8286954a5214 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-801f9095-8204-46df-b2be-8286954a5214 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:12.582548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:12.583786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:12.584833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 532/1040] 10.222.0.20 () {56 vars in 1105 bytes} [Fri Jul 22 11:02:12 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:12.585819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f9095-8204-46df-b2be-8286954a5214 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:12.594206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f495de-3395-408f-a707-f35015ea83f6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:12.629563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f96e-5c25-4e37-a3fc-7787f83e2dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:12.630230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f96e-5c25-4e37-a3fc-7787f83e2dea None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.630512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f96e-5c25-4e37-a3fc-7787f83e2dea None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.630793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f96e-5c25-4e37-a3fc-7787f83e2dea None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.631534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8f96e-5c25-4e37-a3fc-7787f83e2dea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:12.633561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8f96e-5c25-4e37-a3fc-7787f83e2dea None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:12.643413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:12.653258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 509/1041] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:02:12 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:02:12.654125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f96e-5c25-4e37-a3fc-7787f83e2dea None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:12.693131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f9095-8204-46df-b2be-8286954a5214 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:12.734191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85373cd8-09ee-4455-8055-74cc6c15155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:12.734882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85373cd8-09ee-4455-8055-74cc6c15155f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.735168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85373cd8-09ee-4455-8055-74cc6c15155f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.735479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85373cd8-09ee-4455-8055-74cc6c15155f None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.736347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85373cd8-09ee-4455-8055-74cc6c15155f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:12.736658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85373cd8-09ee-4455-8055-74cc6c15155f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:12.738369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85373cd8-09ee-4455-8055-74cc6c15155f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:12.759615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:12.767124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 533/1042] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 22 11:02:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:12.772843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85373cd8-09ee-4455-8055-74cc6c15155f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:12.794056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f96e-5c25-4e37-a3fc-7787f83e2dea None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:12.837451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d81320-87a7-454d-b19f-33d790d05cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:12.838207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d81320-87a7-454d-b19f-33d790d05cca None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.838491 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d81320-87a7-454d-b19f-33d790d05cca None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.838772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d81320-87a7-454d-b19f-33d790d05cca None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.839619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d81320-87a7-454d-b19f-33d790d05cca None admin] RBAC: Adding query filter params (name=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:12.839930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d81320-87a7-454d-b19f-33d790d05cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:12.841320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d81320-87a7-454d-b19f-33d790d05cca None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:12.847669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 510/1043] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:12.855546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d81320-87a7-454d-b19f-33d790d05cca None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:12.895233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85373cd8-09ee-4455-8055-74cc6c15155f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:12.915173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:12.937722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e61d2-a935-4e8f-9005-8fedfe8757c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:12.938613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e61d2-a935-4e8f-9005-8fedfe8757c0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:12.939058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e61d2-a935-4e8f-9005-8fedfe8757c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:12.939551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e61d2-a935-4e8f-9005-8fedfe8757c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:12.965927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d81320-87a7-454d-b19f-33d790d05cca None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:13.001988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b612f8-95be-42f8-8ad5-a1095c162859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:13.002653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b612f8-95be-42f8-8ad5-a1095c162859 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:13.002942 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b612f8-95be-42f8-8ad5-a1095c162859 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:13.003222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b612f8-95be-42f8-8ad5-a1095c162859 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:13.003958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b612f8-95be-42f8-8ad5-a1095c162859 None admin] RBAC: Adding query filter params (type=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:13.004276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b612f8-95be-42f8-8ad5-a1095c162859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:13.005653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b612f8-95be-42f8-8ad5-a1095c162859 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:13.010731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 511/1044] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:13.016244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b612f8-95be-42f8-8ad5-a1095c162859 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:13.027057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:13.031388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e61d2-a935-4e8f-9005-8fedfe8757c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:13.033280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e61d2-a935-4e8f-9005-8fedfe8757c0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:13.122763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b612f8-95be-42f8-8ad5-a1095c162859 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:13.157905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 534/1045] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:12 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:02:13.163392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e61d2-a935-4e8f-9005-8fedfe8757c0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:13.171320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9604337-98d4-405b-b372-5c0c0e3e65ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:13.172000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9604337-98d4-405b-b372-5c0c0e3e65ca None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:13.172292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9604337-98d4-405b-b372-5c0c0e3e65ca None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:13.172573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9604337-98d4-405b-b372-5c0c0e3e65ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:13.174403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9604337-98d4-405b-b372-5c0c0e3e65ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:13.176890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9604337-98d4-405b-b372-5c0c0e3e65ca None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:13.183425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:13.198797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 512/1046] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 22 11:02:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:02:13.278747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e61d2-a935-4e8f-9005-8fedfe8757c0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:13.321363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae544de-5b8c-4d9d-9da2-d32509bd5b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:13.322159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae544de-5b8c-4d9d-9da2-d32509bd5b07 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:13.322451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae544de-5b8c-4d9d-9da2-d32509bd5b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:13.322728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae544de-5b8c-4d9d-9da2-d32509bd5b07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:13.334019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae544de-5b8c-4d9d-9da2-d32509bd5b07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:13.336065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae544de-5b8c-4d9d-9da2-d32509bd5b07 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-aae544de-5b8c-4d9d-9da2-d32509bd5b07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:02:13.355096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:13.356130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:13.357121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:02:13.357121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:02:13.357121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:13.357121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:13.357121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 535/1047] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:13.362034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9604337-98d4-405b-b372-5c0c0e3e65ca None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:13.430421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:13.474797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9604337-98d4-405b-b372-5c0c0e3e65ca None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:13.515312 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd48f2c-5184-4eaa-babc-16ab8b143981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:13.515981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd48f2c-5184-4eaa-babc-16ab8b143981 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:13.516267 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd48f2c-5184-4eaa-babc-16ab8b143981 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:13.516543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd48f2c-5184-4eaa-babc-16ab8b143981 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:13.517273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd48f2c-5184-4eaa-babc-16ab8b143981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:13.517581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd48f2c-5184-4eaa-babc-16ab8b143981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:13.518951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd48f2c-5184-4eaa-babc-16ab8b143981 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:13.536298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 513/1048] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:13.537251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae544de-5b8c-4d9d-9da2-d32509bd5b07 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:13.604679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:13.641022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae544de-5b8c-4d9d-9da2-d32509bd5b07 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:13.676426 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f32885-d9f7-4e20-bb4e-bcf7b0d6b2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:13.677226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f32885-d9f7-4e20-bb4e-bcf7b0d6b2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:13.677515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f32885-d9f7-4e20-bb4e-bcf7b0d6b2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:13.677796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f32885-d9f7-4e20-bb4e-bcf7b0d6b2b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:13.684700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f32885-d9f7-4e20-bb4e-bcf7b0d6b2b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:13.686078 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f32885-d9f7-4e20-bb4e-bcf7b0d6b2b5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a4f32885-d9f7-4e20-bb4e-bcf7b0d6b2b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:02:13.700387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:13.701376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:13.702288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:02:13.702288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:02:13.702288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:02:13.702288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:02:13.702288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:02:13.702288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:02:13.702288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:02:13.702288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:13.703573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 536/1049] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:02:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:13.707378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd48f2c-5184-4eaa-babc-16ab8b143981 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:13.822635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd48f2c-5184-4eaa-babc-16ab8b143981 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:13.848734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:13.860886 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fe0eb-3f6d-4455-8451-bca998de8dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:13.861830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fe0eb-3f6d-4455-8451-bca998de8dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:13.862277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fe0eb-3f6d-4455-8451-bca998de8dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:13.862713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fe0eb-3f6d-4455-8451-bca998de8dd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:13.863985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fe0eb-3f6d-4455-8451-bca998de8dd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:13.864467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fe0eb-3f6d-4455-8451-bca998de8dd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:13.866171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fe0eb-3f6d-4455-8451-bca998de8dd9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:13.885084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 514/1050] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:13.886452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f32885-d9f7-4e20-bb4e-bcf7b0d6b2b5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:13.991166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f32885-d9f7-4e20-bb4e-bcf7b0d6b2b5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:14.017231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:14.026913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7846b-1f27-4079-87a9-4212d068bc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:14.027850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7846b-1f27-4079-87a9-4212d068bc4e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:14.028323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7846b-1f27-4079-87a9-4212d068bc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:14.028796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7846b-1f27-4079-87a9-4212d068bc4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:14.029809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7846b-1f27-4079-87a9-4212d068bc4e None admin] RBAC: Adding query filter params (role.id=19b08ccf876549a4b4f59fd06c217412, scope.project.id=89224d740c7148938c2d803804083a57, user.id=c51929d0ecb047b4bdc405dc86fe7eb6) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:14.030321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7846b-1f27-4079-87a9-4212d068bc4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:14.031961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7846b-1f27-4079-87a9-4212d068bc4e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:14.037483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 537/1051] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:02:13 2022] GET /identity/v3/role_assignments?user.id=c51929d0ecb047b4bdc405dc86fe7eb6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=19b08ccf876549a4b4f59fd06c217412 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:14.809058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91603b-6129-4c46-a8d9-389618da7369 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:14.809598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91603b-6129-4c46-a8d9-389618da7369 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:14.809598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91603b-6129-4c46-a8d9-389618da7369 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:14.811132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 515/1052] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:14.816129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef79349-35b6-47cc-8a57-970c35971950 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:14.816521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef79349-35b6-47cc-8a57-970c35971950 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:14.816704 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef79349-35b6-47cc-8a57-970c35971950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:14.835184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fef79349-35b6-47cc-8a57-970c35971950 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:14.837016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fef79349-35b6-47cc-8a57-970c35971950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:14.905244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 538/1053] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:14.929283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91603b-6129-4c46-a8d9-389618da7369 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:15.031670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91603b-6129-4c46-a8d9-389618da7369 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:15.065191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf93ba-5a68-4972-98bc-1341f5952e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:15.065986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf93ba-5a68-4972-98bc-1341f5952e6f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:15.066274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf93ba-5a68-4972-98bc-1341f5952e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:15.066552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf93ba-5a68-4972-98bc-1341f5952e6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:15.077789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf93ba-5a68-4972-98bc-1341f5952e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7cdf93ba-5a68-4972-98bc-1341f5952e6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:15.085402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:15.086849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:15.087763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:15.092200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 516/1054] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:15.093198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef79349-35b6-47cc-8a57-970c35971950 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:15.195877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef79349-35b6-47cc-8a57-970c35971950 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:15.216456 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:15.231206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c119beb-d52a-47d0-a49a-803270a9e1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:15.231911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c119beb-d52a-47d0-a49a-803270a9e1bd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:15.232204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c119beb-d52a-47d0-a49a-803270a9e1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:15.232489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c119beb-d52a-47d0-a49a-803270a9e1bd None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:15.233239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c119beb-d52a-47d0-a49a-803270a9e1bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:15.233551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c119beb-d52a-47d0-a49a-803270a9e1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:15.234925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c119beb-d52a-47d0-a49a-803270a9e1bd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:15.243648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 539/1055] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:02:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:15.249414 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf93ba-5a68-4972-98bc-1341f5952e6f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:15.259293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e67ee-8711-4be9-8556-e71a20951c07 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:15.259293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e67ee-8711-4be9-8556-e71a20951c07 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:15.259293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e67ee-8711-4be9-8556-e71a20951c07 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:15.260573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 540/1056] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:02:15.265235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30593-bd15-49bb-b131-013322005d03 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:15.265523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30593-bd15-49bb-b131-013322005d03 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:15.265819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30593-bd15-49bb-b131-013322005d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:15.283592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c5e30593-bd15-49bb-b131-013322005d03 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:15.284755 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c5e30593-bd15-49bb-b131-013322005d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:15.297294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:15.356367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf93ba-5a68-4972-98bc-1341f5952e6f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:15.363392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 541/1057] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:02:15.377169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e30593-bd15-49bb-b131-013322005d03 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:15.396044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5b24-f1fd-4954-a2e4-60060080ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:15.396920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5b24-f1fd-4954-a2e4-60060080ed7f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:15.397208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5b24-f1fd-4954-a2e4-60060080ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:15.397487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5b24-f1fd-4954-a2e4-60060080ed7f None admin] PATH_INFO: `/v3/users/8fa7bb2c302c4879901fb2e7771cf89d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:15.412175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee5b24-f1fd-4954-a2e4-60060080ed7f None admin] RBAC: Authorizing `identity:get_user(user_id=8fa7bb2c302c4879901fb2e7771cf89d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:15.414362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee5b24-f1fd-4954-a2e4-60060080ed7f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:15.420740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 517/1058] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:15 2022] GET /identity/v3/users/8fa7bb2c302c4879901fb2e7771cf89d => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:15.421851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5b24-f1fd-4954-a2e4-60060080ed7f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:15.481771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e30593-bd15-49bb-b131-013322005d03 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:15.497105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:15.519325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e781fd0-5012-400a-a7c1-e56b328c8380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:15.519980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e781fd0-5012-400a-a7c1-e56b328c8380 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:15.520270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e781fd0-5012-400a-a7c1-e56b328c8380 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:15.520548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e781fd0-5012-400a-a7c1-e56b328c8380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:15.530793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5b24-f1fd-4954-a2e4-60060080ed7f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:15.532818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:15.573746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04349cb-aa90-49d6-a7d7-81111a7483bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:15.574395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04349cb-aa90-49d6-a7d7-81111a7483bf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:15.575063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04349cb-aa90-49d6-a7d7-81111a7483bf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:15.575405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04349cb-aa90-49d6-a7d7-81111a7483bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:15.590540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e781fd0-5012-400a-a7c1-e56b328c8380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:15.592032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e781fd0-5012-400a-a7c1-e56b328c8380 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:15.657509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04349cb-aa90-49d6-a7d7-81111a7483bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:15.659048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04349cb-aa90-49d6-a7d7-81111a7483bf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:15.701462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 542/1059] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 22 11:02:15.702420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e781fd0-5012-400a-a7c1-e56b328c8380 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:15.716958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:15.761953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 518/1060] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 22 11:02:15.767205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04349cb-aa90-49d6-a7d7-81111a7483bf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:15.769082 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:15.814268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e781fd0-5012-400a-a7c1-e56b328c8380 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:15.852106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8acb1c-1a4d-4dfe-9b9d-3c93e02b8b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:15.853001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8acb1c-1a4d-4dfe-9b9d-3c93e02b8b6c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:15.853292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8acb1c-1a4d-4dfe-9b9d-3c93e02b8b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:15.853572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8acb1c-1a4d-4dfe-9b9d-3c93e02b8b6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:15.864331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8acb1c-1a4d-4dfe-9b9d-3c93e02b8b6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:15.865774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8acb1c-1a4d-4dfe-9b9d-3c93e02b8b6c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:15.877340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04349cb-aa90-49d6-a7d7-81111a7483bf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-7c8acb1c-1a4d-4dfe-9b9d-3c93e02b8b6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:02:15.884019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:15.884926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:15.885831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:02:15.885831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:02:15.885831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:15.885831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:15.885831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 543/1061] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:15.891076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8acb1c-1a4d-4dfe-9b9d-3c93e02b8b6c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:15.915893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045faf48-58c6-4583-99c0-0da5c0c57f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:15.917093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045faf48-58c6-4583-99c0-0da5c0c57f20 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:15.917523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045faf48-58c6-4583-99c0-0da5c0c57f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:15.917921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045faf48-58c6-4583-99c0-0da5c0c57f20 None admin] PATH_INFO: `/v3/projects/ce566e7579494c489967787c644b7ce1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:15.919289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045faf48-58c6-4583-99c0-0da5c0c57f20 None admin] RBAC: Authorizing `identity:get_project(project_id=ce566e7579494c489967787c644b7ce1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:15.921296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045faf48-58c6-4583-99c0-0da5c0c57f20 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:15.928581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 519/1062] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:02:15 2022] GET /identity/v3/projects/ce566e7579494c489967787c644b7ce1 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:15.930456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045faf48-58c6-4583-99c0-0da5c0c57f20 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:15.955223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:15.999623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8acb1c-1a4d-4dfe-9b9d-3c93e02b8b6c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:16.003185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:16.036737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84023d8e-ab67-4c52-8f32-4cebbcf89041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:16.037329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045faf48-58c6-4583-99c0-0da5c0c57f20 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:16.037828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84023d8e-ab67-4c52-8f32-4cebbcf89041 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:16.038274 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84023d8e-ab67-4c52-8f32-4cebbcf89041 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:16.039233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84023d8e-ab67-4c52-8f32-4cebbcf89041 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:16.039988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84023d8e-ab67-4c52-8f32-4cebbcf89041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:16.040302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84023d8e-ab67-4c52-8f32-4cebbcf89041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:16.041670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84023d8e-ab67-4c52-8f32-4cebbcf89041 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:16.061687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 544/1063] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:16.063613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efd2d1-0818-4962-9648-b0fe98eb34a4 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:16.063909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efd2d1-0818-4962-9648-b0fe98eb34a4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:16.064191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efd2d1-0818-4962-9648-b0fe98eb34a4 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:16.065849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 545/1064] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:02:16.072178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6711bd2-cd56-42cb-9c53-bb7929db1f81 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:16.073440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6711bd2-cd56-42cb-9c53-bb7929db1f81 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:16.073723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6711bd2-cd56-42cb-9c53-bb7929db1f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:16.083300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54368913-f1c2-490c-bcc2-1931052660b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:16.083992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54368913-f1c2-490c-bcc2-1931052660b3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:16.084280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54368913-f1c2-490c-bcc2-1931052660b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:16.084560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54368913-f1c2-490c-bcc2-1931052660b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:16.085454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54368913-f1c2-490c-bcc2-1931052660b3 None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=ce566e7579494c489967787c644b7ce1, user.id=8fa7bb2c302c4879901fb2e7771cf89d) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:16.085777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54368913-f1c2-490c-bcc2-1931052660b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:16.087294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54368913-f1c2-490c-bcc2-1931052660b3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:16.092031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d6711bd2-cd56-42cb-9c53-bb7929db1f81 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:16.093558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d6711bd2-cd56-42cb-9c53-bb7929db1f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:16.093688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 520/1065] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:02:15 2022] GET /identity/v3/role_assignments?user.id=8fa7bb2c302c4879901fb2e7771cf89d&scope.project.id=ce566e7579494c489967787c644b7ce1&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:16.183538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 546/1066] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:16.206719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54368913-f1c2-490c-bcc2-1931052660b3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:16.316806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54368913-f1c2-490c-bcc2-1931052660b3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:16.352771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fdfa8-a52f-49f4-a55a-9829f4feb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:16.353596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fdfa8-a52f-49f4-a55a-9829f4feb410 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:16.353884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fdfa8-a52f-49f4-a55a-9829f4feb410 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:16.354164 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fdfa8-a52f-49f4-a55a-9829f4feb410 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:16.356171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:16.367048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fdfa8-a52f-49f4-a55a-9829f4feb410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-fc0fdfa8-a52f-49f4-a55a-9829f4feb410 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:16.375554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:16.376494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:16.377347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 521/1067] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 22 11:02:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:16.382771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6711bd2-cd56-42cb-9c53-bb7929db1f81 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:16.436389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:16.481156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6711bd2-cd56-42cb-9c53-bb7929db1f81 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:16.514235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2cfbc-6ef5-46b4-9673-8d85b24f4fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:16.514912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2cfbc-6ef5-46b4-9673-8d85b24f4fca None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:16.515202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2cfbc-6ef5-46b4-9673-8d85b24f4fca None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:16.515515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2cfbc-6ef5-46b4-9673-8d85b24f4fca None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:16.516268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2cfbc-6ef5-46b4-9673-8d85b24f4fca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:16.516581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2cfbc-6ef5-46b4-9673-8d85b24f4fca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:16.517951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2cfbc-6ef5-46b4-9673-8d85b24f4fca None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:16.526349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 547/1068] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 22 11:02:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:16.532116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fdfa8-a52f-49f4-a55a-9829f4feb410 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:16.633363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fdfa8-a52f-49f4-a55a-9829f4feb410 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:16.666162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9644e9a-d952-488f-b206-72479fe85e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:16.666955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9644e9a-d952-488f-b206-72479fe85e44 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:16.667241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9644e9a-d952-488f-b206-72479fe85e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:16.667555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9644e9a-d952-488f-b206-72479fe85e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:16.677832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9644e9a-d952-488f-b206-72479fe85e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:16.679627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9644e9a-d952-488f-b206-72479fe85e44 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b9644e9a-d952-488f-b206-72479fe85e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:02:16.697545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:16.698457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:16.699399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:02:16.699399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:02:16.699399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:16.699399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:16.699399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 522/1069] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:16.704792 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2cfbc-6ef5-46b4-9673-8d85b24f4fca None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:16.806003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2cfbc-6ef5-46b4-9673-8d85b24f4fca None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:16.823183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:16.841618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5749f-fede-433c-9a5e-a34b872fbfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:16.842286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5749f-fede-433c-9a5e-a34b872fbfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:16.842573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5749f-fede-433c-9a5e-a34b872fbfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:16.842851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5749f-fede-433c-9a5e-a34b872fbfe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:16.843613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5749f-fede-433c-9a5e-a34b872fbfe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:16.843920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5749f-fede-433c-9a5e-a34b872fbfe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:16.845281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5749f-fede-433c-9a5e-a34b872fbfe5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:16.856830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 548/1070] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:16.862496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9644e9a-d952-488f-b206-72479fe85e44 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:16.910126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:16.962539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9644e9a-d952-488f-b206-72479fe85e44 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:16.994980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6f388-34a7-4173-a996-c0919e6420b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:16.995790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6f388-34a7-4173-a996-c0919e6420b9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:16.996078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6f388-34a7-4173-a996-c0919e6420b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:16.996355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6f388-34a7-4173-a996-c0919e6420b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:17.003590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6f388-34a7-4173-a996-c0919e6420b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:17.005029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6f388-34a7-4173-a996-c0919e6420b9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-aed6f388-34a7-4173-a996-c0919e6420b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:02:17.019453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:17.020392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:17.021385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 523/1071] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 22 11:02:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:17.026580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5749f-fede-433c-9a5e-a34b872fbfe5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:17.124866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5749f-fede-433c-9a5e-a34b872fbfe5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:17.158057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a09b70-ad3c-4a87-bfe7-4f012707cc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:17.158729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a09b70-ad3c-4a87-bfe7-4f012707cc52 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:17.159020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a09b70-ad3c-4a87-bfe7-4f012707cc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:17.159328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a09b70-ad3c-4a87-bfe7-4f012707cc52 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:17.160169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a09b70-ad3c-4a87-bfe7-4f012707cc52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:17.160482 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a09b70-ad3c-4a87-bfe7-4f012707cc52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:17.161936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a09b70-ad3c-4a87-bfe7-4f012707cc52 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:17.184212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 549/1072] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 22 11:02:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:17.189794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6f388-34a7-4173-a996-c0919e6420b9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:17.290930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6f388-34a7-4173-a996-c0919e6420b9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:17.297675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:17.324406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed449c-39d6-4f7d-98d5-d7e2aea86920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:17.325193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed449c-39d6-4f7d-98d5-d7e2aea86920 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:17.325485 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed449c-39d6-4f7d-98d5-d7e2aea86920 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:17.325769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed449c-39d6-4f7d-98d5-d7e2aea86920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:17.336007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed449c-39d6-4f7d-98d5-d7e2aea86920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:17.337450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed449c-39d6-4f7d-98d5-d7e2aea86920 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-daed449c-39d6-4f7d-98d5-d7e2aea86920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:02:17.354628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:17.355542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:17.356448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:02:17.356448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:02:17.356448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:17.356448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:17.356448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 524/1073] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:17.361685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a09b70-ad3c-4a87-bfe7-4f012707cc52 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:17.395235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:17.458067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a09b70-ad3c-4a87-bfe7-4f012707cc52 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:17.490863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effc773-860e-4a63-8226-1024c8db7b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:17.492090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effc773-860e-4a63-8226-1024c8db7b86 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:17.492391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effc773-860e-4a63-8226-1024c8db7b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:17.492672 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effc773-860e-4a63-8226-1024c8db7b86 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:17.493386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effc773-860e-4a63-8226-1024c8db7b86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:17.493695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effc773-860e-4a63-8226-1024c8db7b86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:17.495058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effc773-860e-4a63-8226-1024c8db7b86 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:17.506028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 550/1074] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:17.511820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed449c-39d6-4f7d-98d5-d7e2aea86920 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:17.613479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed449c-39d6-4f7d-98d5-d7e2aea86920 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:17.646394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa5239-ab68-4850-937a-7cc1ca5f256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:17.647117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa5239-ab68-4850-937a-7cc1ca5f256f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:17.647438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa5239-ab68-4850-937a-7cc1ca5f256f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:17.647721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa5239-ab68-4850-937a-7cc1ca5f256f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:17.654285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa5239-ab68-4850-937a-7cc1ca5f256f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:17.655774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa5239-ab68-4850-937a-7cc1ca5f256f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0cfa5239-ab68-4850-937a-7cc1ca5f256f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:02:17.670291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:17.671190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:17.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:02:17.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:02:17.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:02:17.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:02:17.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:02:17.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:02:17.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:02:17.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:17.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 525/1075] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:02:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:17.677285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effc773-860e-4a63-8226-1024c8db7b86 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:17.769302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:17.776661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effc773-860e-4a63-8226-1024c8db7b86 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:17.808849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dd508-5e78-465c-a696-89c377b8a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:17.809500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dd508-5e78-465c-a696-89c377b8a274 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:17.809784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dd508-5e78-465c-a696-89c377b8a274 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:17.810058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dd508-5e78-465c-a696-89c377b8a274 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:17.810838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dd508-5e78-465c-a696-89c377b8a274 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:17.811147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dd508-5e78-465c-a696-89c377b8a274 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:17.812612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dd508-5e78-465c-a696-89c377b8a274 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:17.824314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 551/1076] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:02:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:17.830220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa5239-ab68-4850-937a-7cc1ca5f256f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:17.849300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:17.929852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa5239-ab68-4850-937a-7cc1ca5f256f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:17.962931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ac682-2dbe-4b3a-ac8f-65619fe21925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:17.963859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ac682-2dbe-4b3a-ac8f-65619fe21925 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:17.964144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ac682-2dbe-4b3a-ac8f-65619fe21925 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:17.964419 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ac682-2dbe-4b3a-ac8f-65619fe21925 None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/c51929d0ecb047b4bdc405dc86fe7eb6/roles/19b08ccf876549a4b4f59fd06c217412` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:17.966605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ac682-2dbe-4b3a-ac8f-65619fe21925 None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=c51929d0ecb047b4bdc405dc86fe7eb6, role_id=19b08ccf876549a4b4f59fd06c217412)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:17.968103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ac682-2dbe-4b3a-ac8f-65619fe21925 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:17.982662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 526/1077] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:02:17 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/c51929d0ecb047b4bdc405dc86fe7eb6/roles/19b08ccf876549a4b4f59fd06c217412 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:02:18.149518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd419b-ab1b-405a-88c4-da14b876580a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:18.150027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd419b-ab1b-405a-88c4-da14b876580a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:18.150111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd419b-ab1b-405a-88c4-da14b876580a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:18.151547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 552/1078] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:18.156904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21230c87-7bad-4606-9f6e-f97e7fa93221 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:18.157194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21230c87-7bad-4606-9f6e-f97e7fa93221 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:18.157473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21230c87-7bad-4606-9f6e-f97e7fa93221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:18.174144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-21230c87-7bad-4606-9f6e-f97e7fa93221 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:18.175341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21230c87-7bad-4606-9f6e-f97e7fa93221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:18.240360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 527/1079] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:18.263221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd419b-ab1b-405a-88c4-da14b876580a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:18.361752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd419b-ab1b-405a-88c4-da14b876580a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:18.395086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6184c5-26e3-4d41-ae5a-e8a95ef3db8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:18.395902 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6184c5-26e3-4d41-ae5a-e8a95ef3db8b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:18.396195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6184c5-26e3-4d41-ae5a-e8a95ef3db8b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:18.396477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6184c5-26e3-4d41-ae5a-e8a95ef3db8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:18.407201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6184c5-26e3-4d41-ae5a-e8a95ef3db8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0f6184c5-26e3-4d41-ae5a-e8a95ef3db8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:18.414893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:18.415883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:18.416732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 553/1080] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:02:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:18.422110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21230c87-7bad-4606-9f6e-f97e7fa93221 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:18.432756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:18.519993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21230c87-7bad-4606-9f6e-f97e7fa93221 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:18.552437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298943b8-1ee8-4902-a5da-96d290d1cf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:18.553104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298943b8-1ee8-4902-a5da-96d290d1cf71 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:18.553391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298943b8-1ee8-4902-a5da-96d290d1cf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:18.553672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298943b8-1ee8-4902-a5da-96d290d1cf71 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:18.554949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298943b8-1ee8-4902-a5da-96d290d1cf71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:18.555281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298943b8-1ee8-4902-a5da-96d290d1cf71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:18.556651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298943b8-1ee8-4902-a5da-96d290d1cf71 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:18.564562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 528/1081] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:02:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:18.571034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6184c5-26e3-4d41-ae5a-e8a95ef3db8b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:18.587042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:18.674151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6184c5-26e3-4d41-ae5a-e8a95ef3db8b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:18.707443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42870e0b-8c8d-4cee-8bb7-147775afd56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:18.708319 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42870e0b-8c8d-4cee-8bb7-147775afd56f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:18.708607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42870e0b-8c8d-4cee-8bb7-147775afd56f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:18.708887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42870e0b-8c8d-4cee-8bb7-147775afd56f None admin] PATH_INFO: `/v3/users/8fa7bb2c302c4879901fb2e7771cf89d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:18.710076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42870e0b-8c8d-4cee-8bb7-147775afd56f None admin] RBAC: Authorizing `identity:get_user(user_id=8fa7bb2c302c4879901fb2e7771cf89d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:18.711704 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42870e0b-8c8d-4cee-8bb7-147775afd56f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:18.713383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 554/1082] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:18 2022] GET /identity/v3/users/8fa7bb2c302c4879901fb2e7771cf89d => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:18.719113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298943b8-1ee8-4902-a5da-96d290d1cf71 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:18.787502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:18.818405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298943b8-1ee8-4902-a5da-96d290d1cf71 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:18.850306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb406174-0851-474a-95d2-361e2e05bde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:18.851706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb406174-0851-474a-95d2-361e2e05bde3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:18.851993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb406174-0851-474a-95d2-361e2e05bde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:18.852272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb406174-0851-474a-95d2-361e2e05bde3 None admin] PATH_INFO: `/v3/projects/ce566e7579494c489967787c644b7ce1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:18.853443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb406174-0851-474a-95d2-361e2e05bde3 None admin] RBAC: Authorizing `identity:get_project(project_id=ce566e7579494c489967787c644b7ce1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:18.854873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb406174-0851-474a-95d2-361e2e05bde3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:18.856752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 529/1083] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:02:18 2022] GET /identity/v3/projects/ce566e7579494c489967787c644b7ce1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:18.862563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42870e0b-8c8d-4cee-8bb7-147775afd56f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:18.944526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:18.962909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42870e0b-8c8d-4cee-8bb7-147775afd56f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:18.996030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb3f89-b652-4f59-b456-31949175a18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:18.996909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb3f89-b652-4f59-b456-31949175a18e None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:18.997189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb3f89-b652-4f59-b456-31949175a18e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:18.997461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb3f89-b652-4f59-b456-31949175a18e None admin] PATH_INFO: `/v3/projects/ce566e7579494c489967787c644b7ce1/users/8fa7bb2c302c4879901fb2e7771cf89d/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:18.999398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb3f89-b652-4f59-b456-31949175a18e None admin] RBAC: Authorizing `identity:create_grant(project_id=ce566e7579494c489967787c644b7ce1, user_id=8fa7bb2c302c4879901fb2e7771cf89d, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:19.000825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb3f89-b652-4f59-b456-31949175a18e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:19.013676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 555/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:02:18 2022] PUT /identity/v3/projects/ce566e7579494c489967787c644b7ce1/users/8fa7bb2c302c4879901fb2e7771cf89d/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:02:19.994423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85907f7e-a5c7-452f-a6b0-23c818758de0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:19.994966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85907f7e-a5c7-452f-a6b0-23c818758de0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:19.995061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85907f7e-a5c7-452f-a6b0-23c818758de0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:19.996433 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 530/1085] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:20.001722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cd356-f173-413c-8539-436363aa68ab None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:20.002120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cd356-f173-413c-8539-436363aa68ab None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:20.002303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cd356-f173-413c-8539-436363aa68ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:20.019172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ff1cd356-f173-413c-8539-436363aa68ab None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:20.020384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ff1cd356-f173-413c-8539-436363aa68ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:20.085111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 556/1086] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:20.110213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85907f7e-a5c7-452f-a6b0-23c818758de0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:20.210551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85907f7e-a5c7-452f-a6b0-23c818758de0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:20.245154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f32db5-05bd-4625-95c8-f5dc064490c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:20.245884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f32db5-05bd-4625-95c8-f5dc064490c3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:20.246163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f32db5-05bd-4625-95c8-f5dc064490c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:20.246437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f32db5-05bd-4625-95c8-f5dc064490c3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:20.253279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:20.258552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f32db5-05bd-4625-95c8-f5dc064490c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-09f32db5-05bd-4625-95c8-f5dc064490c3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:20.266901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:20.267950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:20.268869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 531/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Jul 22 11:02:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:20.274572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cd356-f173-413c-8539-436363aa68ab None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:20.342063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:20.372948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cd356-f173-413c-8539-436363aa68ab None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:20.405234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656a9ed-a39d-4728-a609-cac699e2e23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:20.405897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656a9ed-a39d-4728-a609-cac699e2e23b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:20.406189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656a9ed-a39d-4728-a609-cac699e2e23b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:20.406473 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656a9ed-a39d-4728-a609-cac699e2e23b None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:20.407218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656a9ed-a39d-4728-a609-cac699e2e23b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:20.407562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656a9ed-a39d-4728-a609-cac699e2e23b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:20.408932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656a9ed-a39d-4728-a609-cac699e2e23b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:20.417272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 557/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 22 11:02:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:20.422796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f32db5-05bd-4625-95c8-f5dc064490c3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:20.519644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f32db5-05bd-4625-95c8-f5dc064490c3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:20.552430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f63fed-176a-450c-97a1-ceb94c6e64dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:20.553221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63fed-176a-450c-97a1-ceb94c6e64dd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:20.553514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63fed-176a-450c-97a1-ceb94c6e64dd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:20.553797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63fed-176a-450c-97a1-ceb94c6e64dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:20.564197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f63fed-176a-450c-97a1-ceb94c6e64dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:20.565629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f63fed-176a-450c-97a1-ceb94c6e64dd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-95f63fed-176a-450c-97a1-ceb94c6e64dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:02:20.582633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:20.583760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:20.584688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:02:20.584688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:02:20.584688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:20.584688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:20.584688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 532/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:20.589953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656a9ed-a39d-4728-a609-cac699e2e23b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:20.695507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656a9ed-a39d-4728-a609-cac699e2e23b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:20.729219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22409ed0-fe9f-49fd-a817-a86ef15a98e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:20.729881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22409ed0-fe9f-49fd-a817-a86ef15a98e1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:20.730173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22409ed0-fe9f-49fd-a817-a86ef15a98e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:20.730462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22409ed0-fe9f-49fd-a817-a86ef15a98e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:20.731184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22409ed0-fe9f-49fd-a817-a86ef15a98e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:20.731527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22409ed0-fe9f-49fd-a817-a86ef15a98e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:20.732887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22409ed0-fe9f-49fd-a817-a86ef15a98e1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:20.734333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:20.746283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 558/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:20.751902 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f63fed-176a-450c-97a1-ceb94c6e64dd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:20.801103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:20.849739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f63fed-176a-450c-97a1-ceb94c6e64dd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:20.882845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8ab59-11f5-46b5-ae1f-ee745399d25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:20.883666 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ab59-11f5-46b5-ae1f-ee745399d25f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:20.883952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ab59-11f5-46b5-ae1f-ee745399d25f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:20.884228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ab59-11f5-46b5-ae1f-ee745399d25f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:20.891720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ab59-11f5-46b5-ae1f-ee745399d25f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:20.893151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ab59-11f5-46b5-ae1f-ee745399d25f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-41c8ab59-11f5-46b5-ae1f-ee745399d25f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:02:20.907220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:20.908357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 22 11:02:20.909544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:20.911397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 533/1091] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Jul 22 11:02:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:20.914749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22409ed0-fe9f-49fd-a817-a86ef15a98e1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.010190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22409ed0-fe9f-49fd-a817-a86ef15a98e1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.029539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e5dd4-ad61-4645-8371-453b01719526 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.030024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e5dd4-ad61-4645-8371-453b01719526 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.030123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e5dd4-ad61-4645-8371-453b01719526 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.031506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 534/1092] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:02:21.036399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f7f0b-71c3-4671-a949-1af0a39cf28d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.036689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f7f0b-71c3-4671-a949-1af0a39cf28d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.036967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f7f0b-71c3-4671-a949-1af0a39cf28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.044645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1da01f-834e-4451-b6e8-1f4a6afd86f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.045305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1da01f-834e-4451-b6e8-1f4a6afd86f4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.045590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1da01f-834e-4451-b6e8-1f4a6afd86f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.045866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1da01f-834e-4451-b6e8-1f4a6afd86f4 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.046695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1da01f-834e-4451-b6e8-1f4a6afd86f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:21.047001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1da01f-834e-4451-b6e8-1f4a6afd86f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:21.048466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1da01f-834e-4451-b6e8-1f4a6afd86f4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:21.053853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d60f7f0b-71c3-4671-a949-1af0a39cf28d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:21.054967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d60f7f0b-71c3-4671-a949-1af0a39cf28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:21.066017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 559/1093] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Jul 22 11:02:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:21.072248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1da01f-834e-4451-b6e8-1f4a6afd86f4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.121210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 535/1094] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:02:21.123763 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:21.145976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f7f0b-71c3-4671-a949-1af0a39cf28d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.172565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1da01f-834e-4451-b6e8-1f4a6afd86f4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.186393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:21.205802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a1793-c644-4010-9f53-9268e96cf191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.206451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a1793-c644-4010-9f53-9268e96cf191 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.206736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a1793-c644-4010-9f53-9268e96cf191 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.207013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a1793-c644-4010-9f53-9268e96cf191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.244802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f7f0b-71c3-4671-a949-1af0a39cf28d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.273349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a1793-c644-4010-9f53-9268e96cf191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:21.274790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a1793-c644-4010-9f53-9268e96cf191 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:21.278175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d5221-8520-4a05-b9d5-cba88b94a20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.278959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d5221-8520-4a05-b9d5-cba88b94a20d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.279242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d5221-8520-4a05-b9d5-cba88b94a20d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.279612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d5221-8520-4a05-b9d5-cba88b94a20d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.289790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d5221-8520-4a05-b9d5-cba88b94a20d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-224d5221-8520-4a05-b9d5-cba88b94a20d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:21.297501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:21.298450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:21.299374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 536/1095] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:21.304817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d5221-8520-4a05-b9d5-cba88b94a20d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.326919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:21.373393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 560/1096] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:02:21.379533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a1793-c644-4010-9f53-9268e96cf191 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.406075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:21.409825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d5221-8520-4a05-b9d5-cba88b94a20d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.442649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e3615-a8a0-44b7-8f7c-705eb8de6500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.443348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e3615-a8a0-44b7-8f7c-705eb8de6500 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.443637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e3615-a8a0-44b7-8f7c-705eb8de6500 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.443914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e3615-a8a0-44b7-8f7c-705eb8de6500 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.445212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e3615-a8a0-44b7-8f7c-705eb8de6500 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:21.445519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e3615-a8a0-44b7-8f7c-705eb8de6500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:21.446888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e3615-a8a0-44b7-8f7c-705eb8de6500 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:21.455088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 537/1097] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:21.460753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e3615-a8a0-44b7-8f7c-705eb8de6500 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.474904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a1793-c644-4010-9f53-9268e96cf191 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.508760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd305db7-17bc-42ec-836c-0919f49b241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.509546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd305db7-17bc-42ec-836c-0919f49b241d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.509833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd305db7-17bc-42ec-836c-0919f49b241d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.510112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd305db7-17bc-42ec-836c-0919f49b241d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.519823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd305db7-17bc-42ec-836c-0919f49b241d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:21.521259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd305db7-17bc-42ec-836c-0919f49b241d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fd305db7-17bc-42ec-836c-0919f49b241d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:02:21.538341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:21.539294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:21.540224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:02:21.540224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:02:21.540224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:02:21.540224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:21.540224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 561/1098] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:21.545075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd305db7-17bc-42ec-836c-0919f49b241d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.549419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:21.558571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e3615-a8a0-44b7-8f7c-705eb8de6500 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.592129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706b037-607c-4281-921d-91ac73b6cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.593251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706b037-607c-4281-921d-91ac73b6cb47 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.593540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706b037-607c-4281-921d-91ac73b6cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.593968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706b037-607c-4281-921d-91ac73b6cb47 None admin] PATH_INFO: `/v3/users/8fa7bb2c302c4879901fb2e7771cf89d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.595472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706b037-607c-4281-921d-91ac73b6cb47 None admin] RBAC: Authorizing `identity:get_user(user_id=8fa7bb2c302c4879901fb2e7771cf89d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:21.596917 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706b037-607c-4281-921d-91ac73b6cb47 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:21.599313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 538/1099] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/users/8fa7bb2c302c4879901fb2e7771cf89d => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:21.604843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706b037-607c-4281-921d-91ac73b6cb47 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.636662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:21.645158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd305db7-17bc-42ec-836c-0919f49b241d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.677774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80972d52-92de-4624-b5bd-80259d1ef721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.678434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80972d52-92de-4624-b5bd-80259d1ef721 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.678723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80972d52-92de-4624-b5bd-80259d1ef721 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.679532 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80972d52-92de-4624-b5bd-80259d1ef721 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.680246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80972d52-92de-4624-b5bd-80259d1ef721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:21.680552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80972d52-92de-4624-b5bd-80259d1ef721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:21.681910 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80972d52-92de-4624-b5bd-80259d1ef721 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:21.698103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 562/1100] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:21.699125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80972d52-92de-4624-b5bd-80259d1ef721 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.706506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706b037-607c-4281-921d-91ac73b6cb47 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.739321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502ba32-84ef-4af0-acc1-05db55afd2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.739975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502ba32-84ef-4af0-acc1-05db55afd2db None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.740262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502ba32-84ef-4af0-acc1-05db55afd2db None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.740541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502ba32-84ef-4af0-acc1-05db55afd2db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.760526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:21.798501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80972d52-92de-4624-b5bd-80259d1ef721 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.811706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502ba32-84ef-4af0-acc1-05db55afd2db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:21.813103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502ba32-84ef-4af0-acc1-05db55afd2db None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:21.830908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f58c2-6933-4b33-b192-8986559474d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.831729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f58c2-6933-4b33-b192-8986559474d1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.832021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f58c2-6933-4b33-b192-8986559474d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.832301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f58c2-6933-4b33-b192-8986559474d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.839023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f58c2-6933-4b33-b192-8986559474d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:21.840513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f58c2-6933-4b33-b192-8986559474d1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:21.841123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-7e0f58c2-6933-4b33-b192-8986559474d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:02:21.855507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:21.856613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:21.857719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:02:21.857719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:02:21.857719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:02:21.857719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:02:21.857719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:02:21.857719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:02:21.857719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:02:21.857719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:21.857719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 563/1101] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:02:21.862401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f58c2-6933-4b33-b192-8986559474d1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.914632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 539/1102] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 22 11:02:21.920564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502ba32-84ef-4af0-acc1-05db55afd2db None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:21.959074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f58c2-6933-4b33-b192-8986559474d1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:21.982048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:21.993582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d09304-7dd4-4da9-8ad6-15e0ea0d13bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:21.994235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d09304-7dd4-4da9-8ad6-15e0ea0d13bc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:21.994521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d09304-7dd4-4da9-8ad6-15e0ea0d13bc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:21.994802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d09304-7dd4-4da9-8ad6-15e0ea0d13bc None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:21.995646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d09304-7dd4-4da9-8ad6-15e0ea0d13bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:21.995958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d09304-7dd4-4da9-8ad6-15e0ea0d13bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:21.997384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d09304-7dd4-4da9-8ad6-15e0ea0d13bc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:22.008575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 564/1103] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:22.014454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d09304-7dd4-4da9-8ad6-15e0ea0d13bc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:22.017738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502ba32-84ef-4af0-acc1-05db55afd2db None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:22.046861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:22.052998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d040ff-ce0d-4aa9-84c2-bc6fd03d969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:22.054402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d040ff-ce0d-4aa9-84c2-bc6fd03d969c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:22.054687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d040ff-ce0d-4aa9-84c2-bc6fd03d969c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:22.054962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d040ff-ce0d-4aa9-84c2-bc6fd03d969c None admin] PATH_INFO: `/v3/projects/ce566e7579494c489967787c644b7ce1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:22.056536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d040ff-ce0d-4aa9-84c2-bc6fd03d969c None admin] RBAC: Authorizing `identity:get_project(project_id=ce566e7579494c489967787c644b7ce1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:22.057968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d040ff-ce0d-4aa9-84c2-bc6fd03d969c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:22.059875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 540/1104] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:02:21 2022] GET /identity/v3/projects/ce566e7579494c489967787c644b7ce1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:22.065875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d040ff-ce0d-4aa9-84c2-bc6fd03d969c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:22.111508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d09304-7dd4-4da9-8ad6-15e0ea0d13bc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:22.145767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006e3c0-bd45-4828-bc1a-48201c98cc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:22.146414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006e3c0-bd45-4828-bc1a-48201c98cc76 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:22.146698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006e3c0-bd45-4828-bc1a-48201c98cc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:22.146977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006e3c0-bd45-4828-bc1a-48201c98cc76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:22.147848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006e3c0-bd45-4828-bc1a-48201c98cc76 None admin] RBAC: Adding query filter params (role.id=19b08ccf876549a4b4f59fd06c217412, scope.project.id=89224d740c7148938c2d803804083a57, user.id=c51929d0ecb047b4bdc405dc86fe7eb6) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:22.148160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006e3c0-bd45-4828-bc1a-48201c98cc76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:22.149602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006e3c0-bd45-4828-bc1a-48201c98cc76 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:22.155754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 565/1105] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:02:22 2022] GET /identity/v3/role_assignments?user.id=c51929d0ecb047b4bdc405dc86fe7eb6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=19b08ccf876549a4b4f59fd06c217412 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:22.162740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d040ff-ce0d-4aa9-84c2-bc6fd03d969c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:22.200222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d093698-4ca6-49f2-8397-bf9804c32235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:22.200889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d093698-4ca6-49f2-8397-bf9804c32235 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:22.201173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d093698-4ca6-49f2-8397-bf9804c32235 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:22.201447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d093698-4ca6-49f2-8397-bf9804c32235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:22.202299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d093698-4ca6-49f2-8397-bf9804c32235 None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=ce566e7579494c489967787c644b7ce1, user.id=8fa7bb2c302c4879901fb2e7771cf89d) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:22.202614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d093698-4ca6-49f2-8397-bf9804c32235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:22.204140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d093698-4ca6-49f2-8397-bf9804c32235 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:22.211368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 541/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:02:22 2022] GET /identity/v3/role_assignments?user.id=8fa7bb2c302c4879901fb2e7771cf89d&scope.project.id=ce566e7579494c489967787c644b7ce1&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:02:24.249782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e076102-8184-4661-8cfc-befd7aa224a8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:24.250338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e076102-8184-4661-8cfc-befd7aa224a8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:24.250338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e076102-8184-4661-8cfc-befd7aa224a8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:24.251756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 566/1107] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:24.257339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce2a2f-5517-4261-a7ce-11aebc16f1c3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:24.257731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce2a2f-5517-4261-a7ce-11aebc16f1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:24.257897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce2a2f-5517-4261-a7ce-11aebc16f1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:24.274741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-70ce2a2f-5517-4261-a7ce-11aebc16f1c3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:24.275949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-70ce2a2f-5517-4261-a7ce-11aebc16f1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:24.329981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:24.341112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 542/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:02:24.365235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e076102-8184-4661-8cfc-befd7aa224a8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:24.414481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:24.464262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e076102-8184-4661-8cfc-befd7aa224a8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:24.496668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50845aa8-8ce0-427f-ba27-8375cbfcc002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:24.497439 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50845aa8-8ce0-427f-ba27-8375cbfcc002 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:24.497724 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50845aa8-8ce0-427f-ba27-8375cbfcc002 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:24.498003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50845aa8-8ce0-427f-ba27-8375cbfcc002 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:24.499143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50845aa8-8ce0-427f-ba27-8375cbfcc002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:24.500641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50845aa8-8ce0-427f-ba27-8375cbfcc002 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:24.502176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 567/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:24.508505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce2a2f-5517-4261-a7ce-11aebc16f1c3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:24.603863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce2a2f-5517-4261-a7ce-11aebc16f1c3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:24.637242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4294b0c-4cb3-4061-a2a7-50a59400a683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:24.637899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4294b0c-4cb3-4061-a2a7-50a59400a683 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:24.638182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4294b0c-4cb3-4061-a2a7-50a59400a683 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:24.638458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4294b0c-4cb3-4061-a2a7-50a59400a683 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:24.639165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4294b0c-4cb3-4061-a2a7-50a59400a683 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:24.640621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4294b0c-4cb3-4061-a2a7-50a59400a683 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:24.659321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 543/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:02:24 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:02:26.706018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10124043-5434-4986-87bd-5bd36b7b09ad None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:26.706559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10124043-5434-4986-87bd-5bd36b7b09ad None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:26.706559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10124043-5434-4986-87bd-5bd36b7b09ad None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:26.707969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 568/1111] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:26.712992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd3d2f-3a96-4fd1-ab22-6ea69f97dc00 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:26.713421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd3d2f-3a96-4fd1-ab22-6ea69f97dc00 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:26.713568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd3d2f-3a96-4fd1-ab22-6ea69f97dc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:26.729994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e6bd3d2f-3a96-4fd1-ab22-6ea69f97dc00 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:26.731195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e6bd3d2f-3a96-4fd1-ab22-6ea69f97dc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:26.774237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:26.798004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 544/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:26.821399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10124043-5434-4986-87bd-5bd36b7b09ad None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:26.920735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10124043-5434-4986-87bd-5bd36b7b09ad None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:26.937845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:26.959619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b4127-ef66-4e5c-bee9-dadb72388523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:26.960517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b4127-ef66-4e5c-bee9-dadb72388523 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:26.960809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b4127-ef66-4e5c-bee9-dadb72388523 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:26.961092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b4127-ef66-4e5c-bee9-dadb72388523 None admin] PATH_INFO: `/v3/roles/2e875b35d6744573ae0287b954108209` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:26.962618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b4127-ef66-4e5c-bee9-dadb72388523 None admin] RBAC: Authorizing `identity:get_role(role_id=2e875b35d6744573ae0287b954108209)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:26.964028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b4127-ef66-4e5c-bee9-dadb72388523 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:26.965431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 569/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:26 2022] GET /identity/v3/roles/2e875b35d6744573ae0287b954108209 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:26.971825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd3d2f-3a96-4fd1-ab22-6ea69f97dc00 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:27.070575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd3d2f-3a96-4fd1-ab22-6ea69f97dc00 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:27.103129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50d257-2acb-407d-814a-cd4008fa0917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:27.104025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50d257-2acb-407d-814a-cd4008fa0917 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:27.104309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50d257-2acb-407d-814a-cd4008fa0917 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:27.104586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50d257-2acb-407d-814a-cd4008fa0917 None admin] PATH_INFO: `/v3/users/9cc91a5d85c7480582f48a7e9911f931` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:27.118864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50d257-2acb-407d-814a-cd4008fa0917 None admin] RBAC: Authorizing `identity:get_user(user_id=9cc91a5d85c7480582f48a7e9911f931)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:27.120613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50d257-2acb-407d-814a-cd4008fa0917 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:27.122274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 545/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:26 2022] GET /identity/v3/users/9cc91a5d85c7480582f48a7e9911f931 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:27.128151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b4127-ef66-4e5c-bee9-dadb72388523 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:27.224434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b4127-ef66-4e5c-bee9-dadb72388523 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:27.257111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e65ca-82fc-4cae-97ef-912043b151d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:27.257753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e65ca-82fc-4cae-97ef-912043b151d8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:27.258040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e65ca-82fc-4cae-97ef-912043b151d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:27.258317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e65ca-82fc-4cae-97ef-912043b151d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:27.307873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:27.323185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e65ca-82fc-4cae-97ef-912043b151d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:27.324673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e65ca-82fc-4cae-97ef-912043b151d8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:27.418041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 570/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:02:27.424050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50d257-2acb-407d-814a-cd4008fa0917 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:27.457781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:27.520430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50d257-2acb-407d-814a-cd4008fa0917 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:27.552931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e118028-021d-4d27-a450-ed9dff02b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:27.553795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e118028-021d-4d27-a450-ed9dff02b222 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:27.554079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e118028-021d-4d27-a450-ed9dff02b222 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:27.554354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e118028-021d-4d27-a450-ed9dff02b222 None admin] PATH_INFO: `/v3/projects/ce566e7579494c489967787c644b7ce1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:27.555536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e118028-021d-4d27-a450-ed9dff02b222 None admin] RBAC: Authorizing `identity:get_project(project_id=ce566e7579494c489967787c644b7ce1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:27.556954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e118028-021d-4d27-a450-ed9dff02b222 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:27.558628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 546/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:02:27 2022] GET /identity/v3/projects/ce566e7579494c489967787c644b7ce1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:27.564638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e65ca-82fc-4cae-97ef-912043b151d8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:27.643878 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:27.659891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e65ca-82fc-4cae-97ef-912043b151d8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:27.692333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51314b-d7f6-45ff-9a1f-d06b522d5856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:27.692972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51314b-d7f6-45ff-9a1f-d06b522d5856 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:27.693256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51314b-d7f6-45ff-9a1f-d06b522d5856 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:27.693530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51314b-d7f6-45ff-9a1f-d06b522d5856 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:27.694379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51314b-d7f6-45ff-9a1f-d06b522d5856 None admin] RBAC: Adding query filter params (role.id=2e875b35d6744573ae0287b954108209, scope.project.id=ce566e7579494c489967787c644b7ce1, user.id=9cc91a5d85c7480582f48a7e9911f931) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:27.694687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51314b-d7f6-45ff-9a1f-d06b522d5856 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:27.696143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51314b-d7f6-45ff-9a1f-d06b522d5856 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:27.701410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 571/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:02:27 2022] GET /identity/v3/role_assignments?user.id=9cc91a5d85c7480582f48a7e9911f931&scope.project.id=ce566e7579494c489967787c644b7ce1&role.id=2e875b35d6744573ae0287b954108209 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:29.691833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e21037-b6fd-402c-8b08-b6d16f012bd3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:29.692370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e21037-b6fd-402c-8b08-b6d16f012bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:29.692463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e21037-b6fd-402c-8b08-b6d16f012bd3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:29.693774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 547/1118] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:29.699492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cafc6-9a6a-4b4d-82bd-85b182fdd76f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:29.699886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cafc6-9a6a-4b4d-82bd-85b182fdd76f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:29.700055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cafc6-9a6a-4b4d-82bd-85b182fdd76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:29.716180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0a0cafc6-9a6a-4b4d-82bd-85b182fdd76f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:29.717344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0a0cafc6-9a6a-4b4d-82bd-85b182fdd76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:29.780538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 572/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:29.802383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e21037-b6fd-402c-8b08-b6d16f012bd3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:29.901623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e21037-b6fd-402c-8b08-b6d16f012bd3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:29.903848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:29.934617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601732d2-7a5f-4e9d-94f2-92f76c71d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:29.935489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601732d2-7a5f-4e9d-94f2-92f76c71d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:29.935780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601732d2-7a5f-4e9d-94f2-92f76c71d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:29.936058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601732d2-7a5f-4e9d-94f2-92f76c71d6c0 None admin] PATH_INFO: `/v3/roles/2e875b35d6744573ae0287b954108209` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:29.937529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601732d2-7a5f-4e9d-94f2-92f76c71d6c0 None admin] RBAC: Authorizing `identity:get_role(role_id=2e875b35d6744573ae0287b954108209)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:29.938898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601732d2-7a5f-4e9d-94f2-92f76c71d6c0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:29.940303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 548/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:29 2022] GET /identity/v3/roles/2e875b35d6744573ae0287b954108209 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:29.946624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cafc6-9a6a-4b4d-82bd-85b182fdd76f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:30.022178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:30.043017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cafc6-9a6a-4b4d-82bd-85b182fdd76f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:30.075647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea080a68-4970-4942-a2d1-6226552cc03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:30.076505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea080a68-4970-4942-a2d1-6226552cc03f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:30.076789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea080a68-4970-4942-a2d1-6226552cc03f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:30.077067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea080a68-4970-4942-a2d1-6226552cc03f None admin] PATH_INFO: `/v3/users/9cc91a5d85c7480582f48a7e9911f931` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:30.078225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea080a68-4970-4942-a2d1-6226552cc03f None admin] RBAC: Authorizing `identity:get_user(user_id=9cc91a5d85c7480582f48a7e9911f931)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:30.079677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea080a68-4970-4942-a2d1-6226552cc03f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:30.081293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 573/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:29 2022] GET /identity/v3/users/9cc91a5d85c7480582f48a7e9911f931 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:30.087167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601732d2-7a5f-4e9d-94f2-92f76c71d6c0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:30.183704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601732d2-7a5f-4e9d-94f2-92f76c71d6c0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:30.216710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3917c8b-5e41-4c08-bad4-d5b5f57e5f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:30.217550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3917c8b-5e41-4c08-bad4-d5b5f57e5f00 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:30.217832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3917c8b-5e41-4c08-bad4-d5b5f57e5f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:30.218109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3917c8b-5e41-4c08-bad4-d5b5f57e5f00 None admin] PATH_INFO: `/v3/projects/ce566e7579494c489967787c644b7ce1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:30.219281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3917c8b-5e41-4c08-bad4-d5b5f57e5f00 None admin] RBAC: Authorizing `identity:get_project(project_id=ce566e7579494c489967787c644b7ce1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:30.220714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3917c8b-5e41-4c08-bad4-d5b5f57e5f00 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:30.222387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 549/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:02:30 2022] GET /identity/v3/projects/ce566e7579494c489967787c644b7ce1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:30.228905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea080a68-4970-4942-a2d1-6226552cc03f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:30.331501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea080a68-4970-4942-a2d1-6226552cc03f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:30.367240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf3c81-f833-462b-9f65-b3f266374135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:30.368147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf3c81-f833-462b-9f65-b3f266374135 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:30.368433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf3c81-f833-462b-9f65-b3f266374135 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:30.368709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf3c81-f833-462b-9f65-b3f266374135 None admin] PATH_INFO: `/v3/projects/ce566e7579494c489967787c644b7ce1/users/9cc91a5d85c7480582f48a7e9911f931/roles/2e875b35d6744573ae0287b954108209` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:30.370884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf3c81-f833-462b-9f65-b3f266374135 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce566e7579494c489967787c644b7ce1, user_id=9cc91a5d85c7480582f48a7e9911f931, role_id=2e875b35d6744573ae0287b954108209)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:30.372351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf3c81-f833-462b-9f65-b3f266374135 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:30.385010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 574/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:02:30 2022] PUT /identity/v3/projects/ce566e7579494c489967787c644b7ce1/users/9cc91a5d85c7480582f48a7e9911f931/roles/2e875b35d6744573ae0287b954108209 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:02:32.368805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a7aac-0a46-4ab0-8c8f-2a91a78cf864 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:32.369343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a7aac-0a46-4ab0-8c8f-2a91a78cf864 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:32.369343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a7aac-0a46-4ab0-8c8f-2a91a78cf864 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:32.370720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 550/1124] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:32.375897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4c91d-b58c-44da-a7bc-b3933b45af57 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:32.376302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4c91d-b58c-44da-a7bc-b3933b45af57 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:32.376448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4c91d-b58c-44da-a7bc-b3933b45af57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:32.387969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:32.394683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b1c4c91d-b58c-44da-a7bc-b3933b45af57 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:32.396428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b1c4c91d-b58c-44da-a7bc-b3933b45af57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:32.458903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 575/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:32.482188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a7aac-0a46-4ab0-8c8f-2a91a78cf864 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:32.524156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:32.580367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a7aac-0a46-4ab0-8c8f-2a91a78cf864 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:32.613973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee92d4c-b814-459f-8e68-a1a850f89879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:32.614824 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee92d4c-b814-459f-8e68-a1a850f89879 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:32.615109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee92d4c-b814-459f-8e68-a1a850f89879 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:32.615417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee92d4c-b814-459f-8e68-a1a850f89879 None admin] PATH_INFO: `/v3/roles/2e875b35d6744573ae0287b954108209` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:32.617497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee92d4c-b814-459f-8e68-a1a850f89879 None admin] RBAC: Authorizing `identity:get_role(role_id=2e875b35d6744573ae0287b954108209)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:32.618915 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee92d4c-b814-459f-8e68-a1a850f89879 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:32.620280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 551/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:32 2022] GET /identity/v3/roles/2e875b35d6744573ae0287b954108209 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:32.625936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4c91d-b58c-44da-a7bc-b3933b45af57 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:32.743640 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4c91d-b58c-44da-a7bc-b3933b45af57 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:32.775848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedcc52-9858-4e6e-b834-aad88f941483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:32.776725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedcc52-9858-4e6e-b834-aad88f941483 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:32.777010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedcc52-9858-4e6e-b834-aad88f941483 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:32.777287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedcc52-9858-4e6e-b834-aad88f941483 None admin] PATH_INFO: `/v3/users/9cc91a5d85c7480582f48a7e9911f931` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:32.778455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedcc52-9858-4e6e-b834-aad88f941483 None admin] RBAC: Authorizing `identity:get_user(user_id=9cc91a5d85c7480582f48a7e9911f931)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:32.779926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedcc52-9858-4e6e-b834-aad88f941483 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:32.781555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 576/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:32 2022] GET /identity/v3/users/9cc91a5d85c7480582f48a7e9911f931 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:32.787499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee92d4c-b814-459f-8e68-a1a850f89879 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:32.882120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee92d4c-b814-459f-8e68-a1a850f89879 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:32.890800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:32.916100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8522913-cfc2-4691-ab1c-717f82e4403d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:32.916742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8522913-cfc2-4691-ab1c-717f82e4403d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:32.917033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8522913-cfc2-4691-ab1c-717f82e4403d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:32.917317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8522913-cfc2-4691-ab1c-717f82e4403d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:32.982247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8522913-cfc2-4691-ab1c-717f82e4403d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:32.983729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8522913-cfc2-4691-ab1c-717f82e4403d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:33.081389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 552/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:02:33.087959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedcc52-9858-4e6e-b834-aad88f941483 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:33.091656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:33.185602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedcc52-9858-4e6e-b834-aad88f941483 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:33.217956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef649f-7e9e-4b70-8f55-1497f7decd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:33.218804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef649f-7e9e-4b70-8f55-1497f7decd14 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:33.219092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef649f-7e9e-4b70-8f55-1497f7decd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:33.219411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef649f-7e9e-4b70-8f55-1497f7decd14 None admin] PATH_INFO: `/v3/projects/ce566e7579494c489967787c644b7ce1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:33.220837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef649f-7e9e-4b70-8f55-1497f7decd14 None admin] RBAC: Authorizing `identity:get_project(project_id=ce566e7579494c489967787c644b7ce1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:33.222265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef649f-7e9e-4b70-8f55-1497f7decd14 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:33.223984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 577/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:02:33 2022] GET /identity/v3/projects/ce566e7579494c489967787c644b7ce1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:33.230059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8522913-cfc2-4691-ab1c-717f82e4403d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:33.240173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:33.325828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8522913-cfc2-4691-ab1c-717f82e4403d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:33.358229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0a4b4-5144-46e7-8a8f-9e5dc25d97e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:33.358874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0a4b4-5144-46e7-8a8f-9e5dc25d97e6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:33.359159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0a4b4-5144-46e7-8a8f-9e5dc25d97e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:33.359461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0a4b4-5144-46e7-8a8f-9e5dc25d97e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:33.360289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0a4b4-5144-46e7-8a8f-9e5dc25d97e6 None admin] RBAC: Adding query filter params (role.id=2e875b35d6744573ae0287b954108209, scope.project.id=ce566e7579494c489967787c644b7ce1, user.id=9cc91a5d85c7480582f48a7e9911f931) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:33.360603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0a4b4-5144-46e7-8a8f-9e5dc25d97e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:33.362026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0a4b4-5144-46e7-8a8f-9e5dc25d97e6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:33.367973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 553/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:02:33 2022] GET /identity/v3/role_assignments?user.id=9cc91a5d85c7480582f48a7e9911f931&scope.project.id=ce566e7579494c489967787c644b7ce1&role.id=2e875b35d6744573ae0287b954108209 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:35.375551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681090e-4377-4dbe-9179-f99d3b8be762 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:35.376083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681090e-4377-4dbe-9179-f99d3b8be762 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:35.376201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681090e-4377-4dbe-9179-f99d3b8be762 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:35.377481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 578/1131] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:35.382781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b6c10-d971-4aac-b5a5-f9a34664b901 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:35.383204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b6c10-d971-4aac-b5a5-f9a34664b901 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:35.383367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b6c10-d971-4aac-b5a5-f9a34664b901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:35.399587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e08b6c10-d971-4aac-b5a5-f9a34664b901 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:35.400758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e08b6c10-d971-4aac-b5a5-f9a34664b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:35.463779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 554/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:35.487071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681090e-4377-4dbe-9179-f99d3b8be762 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:35.557072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:35.587047 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681090e-4377-4dbe-9179-f99d3b8be762 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:35.622658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7011e7-5dcb-419a-91c2-9bda2e7fa0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:35.623492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7011e7-5dcb-419a-91c2-9bda2e7fa0ab None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:35.623786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7011e7-5dcb-419a-91c2-9bda2e7fa0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:35.624069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7011e7-5dcb-419a-91c2-9bda2e7fa0ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:35.625335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7011e7-5dcb-419a-91c2-9bda2e7fa0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:35.626826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7011e7-5dcb-419a-91c2-9bda2e7fa0ab None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:35.628505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 579/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:35.634253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b6c10-d971-4aac-b5a5-f9a34664b901 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:35.636461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:35.731351 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b6c10-d971-4aac-b5a5-f9a34664b901 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:35.763424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d9d2c-eed9-49de-88a0-65d60ce878c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:35.764074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d9d2c-eed9-49de-88a0-65d60ce878c4 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:35.764359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d9d2c-eed9-49de-88a0-65d60ce878c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:35.764638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d9d2c-eed9-49de-88a0-65d60ce878c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:35.765348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d9d2c-eed9-49de-88a0-65d60ce878c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:35.766777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d9d2c-eed9-49de-88a0-65d60ce878c4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:35.793272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 555/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:02:35 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:02:37.823846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f43bf-b0d3-4b82-bdce-dfb30c927b8d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:37.824410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f43bf-b0d3-4b82-bdce-dfb30c927b8d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:37.824410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f43bf-b0d3-4b82-bdce-dfb30c927b8d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:37.825771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 580/1135] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:37.831012 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a18ca-2c3e-4818-9536-2fd5a19bfa35 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:37.831423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a18ca-2c3e-4818-9536-2fd5a19bfa35 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:37.831621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a18ca-2c3e-4818-9536-2fd5a19bfa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:37.848097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-217a18ca-2c3e-4818-9536-2fd5a19bfa35 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:37.849233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-217a18ca-2c3e-4818-9536-2fd5a19bfa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:37.900024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:37.917954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 556/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:37.942433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f43bf-b0d3-4b82-bdce-dfb30c927b8d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:38.040740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f43bf-b0d3-4b82-bdce-dfb30c927b8d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:38.072528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4e830-d964-4b8b-b20c-cdbd9a195c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:38.073308 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4e830-d964-4b8b-b20c-cdbd9a195c2d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:38.073595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4e830-d964-4b8b-b20c-cdbd9a195c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:38.073873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4e830-d964-4b8b-b20c-cdbd9a195c2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:38.075028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4e830-d964-4b8b-b20c-cdbd9a195c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:38.076485 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4e830-d964-4b8b-b20c-cdbd9a195c2d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:38.078087 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 581/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:02:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:38.083785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a18ca-2c3e-4818-9536-2fd5a19bfa35 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:38.182813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a18ca-2c3e-4818-9536-2fd5a19bfa35 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:38.214938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d85e6-e30d-425f-a921-77eb7c7699f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:38.215691 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d85e6-e30d-425f-a921-77eb7c7699f9 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:38.215975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d85e6-e30d-425f-a921-77eb7c7699f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:38.216248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d85e6-e30d-425f-a921-77eb7c7699f9 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:38.216963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d85e6-e30d-425f-a921-77eb7c7699f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:38.218381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d85e6-e30d-425f-a921-77eb7c7699f9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:38.238039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 557/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:02:38 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:02:40.261908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30c4a9-b10f-4f7b-9b44-5a1c6025968f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:40.262466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30c4a9-b10f-4f7b-9b44-5a1c6025968f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:40.262466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30c4a9-b10f-4f7b-9b44-5a1c6025968f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:40.263936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 582/1139] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:40.269130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddbcfb-2916-4253-b1f0-a0f33fc870cc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:40.269507 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddbcfb-2916-4253-b1f0-a0f33fc870cc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:40.269699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddbcfb-2916-4253-b1f0-a0f33fc870cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:40.287960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bfddbcfb-2916-4253-b1f0-a0f33fc870cc None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:40.289145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bfddbcfb-2916-4253-b1f0-a0f33fc870cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:40.321509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:40.355187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 558/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:40.380823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30c4a9-b10f-4f7b-9b44-5a1c6025968f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:40.391562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:40.480020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30c4a9-b10f-4f7b-9b44-5a1c6025968f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:40.512913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a64e1-d3dd-4321-8e4a-dfe7ffa3f228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:40.513630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a64e1-d3dd-4321-8e4a-dfe7ffa3f228 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:40.513917 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a64e1-d3dd-4321-8e4a-dfe7ffa3f228 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:40.514198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a64e1-d3dd-4321-8e4a-dfe7ffa3f228 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:40.524504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a64e1-d3dd-4321-8e4a-dfe7ffa3f228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ff2a64e1-d3dd-4321-8e4a-dfe7ffa3f228 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:40.532106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:40.533175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:40.534046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 583/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:02:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:40.539786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddbcfb-2916-4253-b1f0-a0f33fc870cc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:40.634723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddbcfb-2916-4253-b1f0-a0f33fc870cc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:40.667238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ea5e-1246-4a18-b898-d7116584fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:40.667926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ea5e-1246-4a18-b898-d7116584fb07 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:40.668213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ea5e-1246-4a18-b898-d7116584fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:40.668492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ea5e-1246-4a18-b898-d7116584fb07 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:40.669239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01ea5e-1246-4a18-b898-d7116584fb07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:40.669551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01ea5e-1246-4a18-b898-d7116584fb07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:40.670926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01ea5e-1246-4a18-b898-d7116584fb07 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:40.678713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 559/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:02:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:40.684389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a64e1-d3dd-4321-8e4a-dfe7ffa3f228 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:40.762173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:40.795059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a64e1-d3dd-4321-8e4a-dfe7ffa3f228 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:40.827336 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68eba3-e192-4e80-816d-af4415b12e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:40.828196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68eba3-e192-4e80-816d-af4415b12e2d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:40.828479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68eba3-e192-4e80-816d-af4415b12e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:40.828753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68eba3-e192-4e80-816d-af4415b12e2d None admin] PATH_INFO: `/v3/users/6908cc8775b046a28f2747a65f3939cc` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:40.841877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68eba3-e192-4e80-816d-af4415b12e2d None admin] RBAC: Authorizing `identity:get_user(user_id=6908cc8775b046a28f2747a65f3939cc)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:40.843331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68eba3-e192-4e80-816d-af4415b12e2d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:40.844952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 584/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:40 2022] GET /identity/v3/users/6908cc8775b046a28f2747a65f3939cc => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:40.850832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ea5e-1246-4a18-b898-d7116584fb07 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:40.882064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:40.950145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ea5e-1246-4a18-b898-d7116584fb07 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:40.985709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1456fc1-1272-43c1-8fd8-58a226754437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:40.986770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1456fc1-1272-43c1-8fd8-58a226754437 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:40.987281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1456fc1-1272-43c1-8fd8-58a226754437 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:40.988071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1456fc1-1272-43c1-8fd8-58a226754437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:41.054812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1456fc1-1272-43c1-8fd8-58a226754437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:41.056228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1456fc1-1272-43c1-8fd8-58a226754437 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:41.089819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:41.159374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 560/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:02:41.165420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68eba3-e192-4e80-816d-af4415b12e2d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:41.261227 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68eba3-e192-4e80-816d-af4415b12e2d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:41.285167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:41.294482 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb66ce-5b10-49d1-9751-a1338b924e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:41.295348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb66ce-5b10-49d1-9751-a1338b924e91 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:41.295638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb66ce-5b10-49d1-9751-a1338b924e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:41.295916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb66ce-5b10-49d1-9751-a1338b924e91 None admin] PATH_INFO: `/v3/projects/3b0e44b548fc4f998f2ab53e8eb28c01` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:41.297073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb66ce-5b10-49d1-9751-a1338b924e91 None admin] RBAC: Authorizing `identity:get_project(project_id=3b0e44b548fc4f998f2ab53e8eb28c01)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:41.298486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb66ce-5b10-49d1-9751-a1338b924e91 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:41.300179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 585/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:02:41 2022] GET /identity/v3/projects/3b0e44b548fc4f998f2ab53e8eb28c01 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:41.306791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1456fc1-1272-43c1-8fd8-58a226754437 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:41.407158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1456fc1-1272-43c1-8fd8-58a226754437 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:41.436346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:41.441751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac3a71-35a6-4457-8c33-8a1dc15c667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:41.442405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac3a71-35a6-4457-8c33-8a1dc15c667a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:41.442695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac3a71-35a6-4457-8c33-8a1dc15c667a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:41.442979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac3a71-35a6-4457-8c33-8a1dc15c667a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:41.443873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac3a71-35a6-4457-8c33-8a1dc15c667a None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=3b0e44b548fc4f998f2ab53e8eb28c01, user.id=6908cc8775b046a28f2747a65f3939cc) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:41.444191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac3a71-35a6-4457-8c33-8a1dc15c667a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:41.445643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac3a71-35a6-4457-8c33-8a1dc15c667a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:41.451037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 561/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:02:41 2022] GET /identity/v3/role_assignments?user.id=6908cc8775b046a28f2747a65f3939cc&scope.project.id=3b0e44b548fc4f998f2ab53e8eb28c01&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:43.448102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3511a-aed5-4518-aaf8-5ef5567d6b24 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:43.448660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3511a-aed5-4518-aaf8-5ef5567d6b24 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:43.448660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3511a-aed5-4518-aaf8-5ef5567d6b24 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:43.450033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 586/1147] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:43.455231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c4d9b-da07-4661-af50-5db9b8963518 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:43.455623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c4d9b-da07-4661-af50-5db9b8963518 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:43.455819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c4d9b-da07-4661-af50-5db9b8963518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:43.472758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cf3c4d9b-da07-4661-af50-5db9b8963518 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:43.473927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cf3c4d9b-da07-4661-af50-5db9b8963518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:43.539131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 562/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:43.561094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3511a-aed5-4518-aaf8-5ef5567d6b24 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:43.660415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3511a-aed5-4518-aaf8-5ef5567d6b24 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:43.692863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf032e25-7fd1-41a3-8f1f-6e3c607dcef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:43.694874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf032e25-7fd1-41a3-8f1f-6e3c607dcef7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:43.695151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf032e25-7fd1-41a3-8f1f-6e3c607dcef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:43.695452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf032e25-7fd1-41a3-8f1f-6e3c607dcef7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:43.710391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf032e25-7fd1-41a3-8f1f-6e3c607dcef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-bf032e25-7fd1-41a3-8f1f-6e3c607dcef7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:43.717970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:43.719008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:43.720231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 587/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:02:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:43.725341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c4d9b-da07-4661-af50-5db9b8963518 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:43.825334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c4d9b-da07-4661-af50-5db9b8963518 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:43.850178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:43.859455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f9350-27bb-4e77-bb16-d36db9f43786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:43.860123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f9350-27bb-4e77-bb16-d36db9f43786 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:43.860411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f9350-27bb-4e77-bb16-d36db9f43786 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:43.860697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f9350-27bb-4e77-bb16-d36db9f43786 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:43.861444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f9350-27bb-4e77-bb16-d36db9f43786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:43.861783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f9350-27bb-4e77-bb16-d36db9f43786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:43.863455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f9350-27bb-4e77-bb16-d36db9f43786 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:43.871536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 563/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:02:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:43.877243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf032e25-7fd1-41a3-8f1f-6e3c607dcef7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:43.931120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:43.973449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf032e25-7fd1-41a3-8f1f-6e3c607dcef7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:44.005479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8796d-0b05-42af-8989-fe73e7193a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:44.006928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8796d-0b05-42af-8989-fe73e7193a84 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:44.007205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8796d-0b05-42af-8989-fe73e7193a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:44.007508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8796d-0b05-42af-8989-fe73e7193a84 None admin] PATH_INFO: `/v3/users/6908cc8775b046a28f2747a65f3939cc` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:44.008666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8796d-0b05-42af-8989-fe73e7193a84 None admin] RBAC: Authorizing `identity:get_user(user_id=6908cc8775b046a28f2747a65f3939cc)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:44.010085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8796d-0b05-42af-8989-fe73e7193a84 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:44.011712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 588/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:43 2022] GET /identity/v3/users/6908cc8775b046a28f2747a65f3939cc => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:44.017342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f9350-27bb-4e77-bb16-d36db9f43786 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:44.127591 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f9350-27bb-4e77-bb16-d36db9f43786 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:44.160953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51ffea-ff79-4b46-aebb-cd97b9ab6d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:44.161819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51ffea-ff79-4b46-aebb-cd97b9ab6d9f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:44.162153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51ffea-ff79-4b46-aebb-cd97b9ab6d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:44.162418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51ffea-ff79-4b46-aebb-cd97b9ab6d9f None admin] PATH_INFO: `/v3/projects/3b0e44b548fc4f998f2ab53e8eb28c01` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:44.163623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51ffea-ff79-4b46-aebb-cd97b9ab6d9f None admin] RBAC: Authorizing `identity:get_project(project_id=3b0e44b548fc4f998f2ab53e8eb28c01)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:44.165076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51ffea-ff79-4b46-aebb-cd97b9ab6d9f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:44.167054 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 564/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:02:44 2022] GET /identity/v3/projects/3b0e44b548fc4f998f2ab53e8eb28c01 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:44.172829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8796d-0b05-42af-8989-fe73e7193a84 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:44.271015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8796d-0b05-42af-8989-fe73e7193a84 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:44.292854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:44.306341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba1d2e-5bc5-4b41-9acf-68f530f3f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:44.307221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba1d2e-5bc5-4b41-9acf-68f530f3f008 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:44.307528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba1d2e-5bc5-4b41-9acf-68f530f3f008 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:44.307805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba1d2e-5bc5-4b41-9acf-68f530f3f008 None admin] PATH_INFO: `/v3/projects/3b0e44b548fc4f998f2ab53e8eb28c01/users/6908cc8775b046a28f2747a65f3939cc/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:44.309709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba1d2e-5bc5-4b41-9acf-68f530f3f008 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b0e44b548fc4f998f2ab53e8eb28c01, user_id=6908cc8775b046a28f2747a65f3939cc, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:44.311126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba1d2e-5bc5-4b41-9acf-68f530f3f008 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:44.323203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 589/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:02:44 2022] PUT /identity/v3/projects/3b0e44b548fc4f998f2ab53e8eb28c01/users/6908cc8775b046a28f2747a65f3939cc/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:02:46.319582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa605ecb-3f57-48a6-b1aa-b8f4bc687baa None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:46.320135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa605ecb-3f57-48a6-b1aa-b8f4bc687baa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:46.320228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa605ecb-3f57-48a6-b1aa-b8f4bc687baa None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:46.321621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 565/1154] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:46.326840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76072a5-bc53-41ce-8299-119e02518030 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:46.327241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76072a5-bc53-41ce-8299-119e02518030 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:46.327444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76072a5-bc53-41ce-8299-119e02518030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:46.344700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b76072a5-bc53-41ce-8299-119e02518030 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:46.345882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b76072a5-bc53-41ce-8299-119e02518030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:46.414924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 590/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:46.437761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa605ecb-3f57-48a6-b1aa-b8f4bc687baa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:46.494532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:46.539925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa605ecb-3f57-48a6-b1aa-b8f4bc687baa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:46.572597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6feb18c-c6d1-41f2-b7fc-e82d8b211f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:46.573324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6feb18c-c6d1-41f2-b7fc-e82d8b211f6d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:46.573612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6feb18c-c6d1-41f2-b7fc-e82d8b211f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:46.573890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6feb18c-c6d1-41f2-b7fc-e82d8b211f6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:46.584805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6feb18c-c6d1-41f2-b7fc-e82d8b211f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f6feb18c-c6d1-41f2-b7fc-e82d8b211f6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:46.592567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:46.593626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:46.594604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 566/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:02:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:46.600046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76072a5-bc53-41ce-8299-119e02518030 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:46.694631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76072a5-bc53-41ce-8299-119e02518030 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:46.707155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:46.728813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfccb3f9-7455-4b70-8bbf-90ab6c42a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:46.729476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccb3f9-7455-4b70-8bbf-90ab6c42a7fb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:46.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccb3f9-7455-4b70-8bbf-90ab6c42a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:46.730046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccb3f9-7455-4b70-8bbf-90ab6c42a7fb None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:46.730790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfccb3f9-7455-4b70-8bbf-90ab6c42a7fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:46.731099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfccb3f9-7455-4b70-8bbf-90ab6c42a7fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:46.732492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfccb3f9-7455-4b70-8bbf-90ab6c42a7fb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:46.740571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 591/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:02:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:46.746430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6feb18c-c6d1-41f2-b7fc-e82d8b211f6d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:46.844905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6feb18c-c6d1-41f2-b7fc-e82d8b211f6d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:46.860696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:46.878579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d87dfd-3d4e-474a-a37f-095c784f38c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:46.879478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d87dfd-3d4e-474a-a37f-095c784f38c6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:46.879767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d87dfd-3d4e-474a-a37f-095c784f38c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:46.880042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d87dfd-3d4e-474a-a37f-095c784f38c6 None admin] PATH_INFO: `/v3/users/6908cc8775b046a28f2747a65f3939cc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:46.881726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d87dfd-3d4e-474a-a37f-095c784f38c6 None admin] RBAC: Authorizing `identity:get_user(user_id=6908cc8775b046a28f2747a65f3939cc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:46.883240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d87dfd-3d4e-474a-a37f-095c784f38c6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:46.884893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 567/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:46 2022] GET /identity/v3/users/6908cc8775b046a28f2747a65f3939cc => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:46.890741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfccb3f9-7455-4b70-8bbf-90ab6c42a7fb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:46.985944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfccb3f9-7455-4b70-8bbf-90ab6c42a7fb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:47.018925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab0699-adb7-45e3-902e-67c74a62fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:47.019589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0699-adb7-45e3-902e-67c74a62fc12 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:47.019872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0699-adb7-45e3-902e-67c74a62fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:47.020147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0699-adb7-45e3-902e-67c74a62fc12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:47.059896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:47.084203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0699-adb7-45e3-902e-67c74a62fc12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:47.085625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0699-adb7-45e3-902e-67c74a62fc12 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:47.180616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 592/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:02:47.186442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d87dfd-3d4e-474a-a37f-095c784f38c6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:47.283236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d87dfd-3d4e-474a-a37f-095c784f38c6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:47.316143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222b2a1-1fb2-4ad7-8e61-54e3bb590fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:47.316989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222b2a1-1fb2-4ad7-8e61-54e3bb590fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:47.317270 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222b2a1-1fb2-4ad7-8e61-54e3bb590fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:47.317544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222b2a1-1fb2-4ad7-8e61-54e3bb590fc2 None admin] PATH_INFO: `/v3/projects/3b0e44b548fc4f998f2ab53e8eb28c01` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:47.318719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222b2a1-1fb2-4ad7-8e61-54e3bb590fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=3b0e44b548fc4f998f2ab53e8eb28c01)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:47.320184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222b2a1-1fb2-4ad7-8e61-54e3bb590fc2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:47.321858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 568/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:02:47 2022] GET /identity/v3/projects/3b0e44b548fc4f998f2ab53e8eb28c01 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:47.327854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab0699-adb7-45e3-902e-67c74a62fc12 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:47.398331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:47.422765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab0699-adb7-45e3-902e-67c74a62fc12 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:47.454652 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921effd1-aa2c-4ea6-a9fd-b88a6ac3400e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:47.455235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921effd1-aa2c-4ea6-a9fd-b88a6ac3400e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:47.455561 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921effd1-aa2c-4ea6-a9fd-b88a6ac3400e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:47.455837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921effd1-aa2c-4ea6-a9fd-b88a6ac3400e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:47.456649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921effd1-aa2c-4ea6-a9fd-b88a6ac3400e None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=3b0e44b548fc4f998f2ab53e8eb28c01, user.id=6908cc8775b046a28f2747a65f3939cc) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:47.456959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921effd1-aa2c-4ea6-a9fd-b88a6ac3400e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:47.458374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921effd1-aa2c-4ea6-a9fd-b88a6ac3400e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:47.463994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 593/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:02:47 2022] GET /identity/v3/role_assignments?user.id=6908cc8775b046a28f2747a65f3939cc&scope.project.id=3b0e44b548fc4f998f2ab53e8eb28c01&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:49.530475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6769a-164f-4bec-ac4f-050d53ce1a97 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:49.531025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6769a-164f-4bec-ac4f-050d53ce1a97 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:49.531025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6769a-164f-4bec-ac4f-050d53ce1a97 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:49.532428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 569/1162] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:49.538426 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f3d5a-ad64-4ce3-b45a-65dc0e926652 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:49.538839 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f3d5a-ad64-4ce3-b45a-65dc0e926652 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:49.538991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f3d5a-ad64-4ce3-b45a-65dc0e926652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:49.555310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-209f3d5a-ad64-4ce3-b45a-65dc0e926652 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:49.556517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-209f3d5a-ad64-4ce3-b45a-65dc0e926652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:49.622996 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 594/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:49.645300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6769a-164f-4bec-ac4f-050d53ce1a97 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:49.698572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:49.745344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6769a-164f-4bec-ac4f-050d53ce1a97 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:49.778753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1094d2c-6252-4603-9c8f-a53a22de530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:49.779420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1094d2c-6252-4603-9c8f-a53a22de530c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:49.779703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1094d2c-6252-4603-9c8f-a53a22de530c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:49.779979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1094d2c-6252-4603-9c8f-a53a22de530c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:49.844977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1094d2c-6252-4603-9c8f-a53a22de530c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:49.846444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1094d2c-6252-4603-9c8f-a53a22de530c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:49.894697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:49.940497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 570/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:02:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:02:49.946932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f3d5a-ad64-4ce3-b45a-65dc0e926652 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:50.010116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:50.041702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f3d5a-ad64-4ce3-b45a-65dc0e926652 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:50.074550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693e53b-fed6-47c8-a45d-f81a61eea1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:50.075369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e53b-fed6-47c8-a45d-f81a61eea1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:50.075677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e53b-fed6-47c8-a45d-f81a61eea1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:50.075942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e53b-fed6-47c8-a45d-f81a61eea1c0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:50.082389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e53b-fed6-47c8-a45d-f81a61eea1c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:50.083885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e53b-fed6-47c8-a45d-f81a61eea1c0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c693e53b-fed6-47c8-a45d-f81a61eea1c0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:50.097670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:50.098858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:02:50.100158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 595/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:02:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:50.104641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1094d2c-6252-4603-9c8f-a53a22de530c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:50.200452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1094d2c-6252-4603-9c8f-a53a22de530c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:50.233222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7b100-2036-4223-9c64-6e44bc8ac7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:50.233880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7b100-2036-4223-9c64-6e44bc8ac7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:50.234170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7b100-2036-4223-9c64-6e44bc8ac7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:50.234452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7b100-2036-4223-9c64-6e44bc8ac7b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:50.235178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7b100-2036-4223-9c64-6e44bc8ac7b1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:50.235520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7b100-2036-4223-9c64-6e44bc8ac7b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:50.236888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7b100-2036-4223-9c64-6e44bc8ac7b1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:50.244744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 571/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Jul 22 11:02:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:52.257687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06186993-4914-43bf-ae43-05199ed9872e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:52.258276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06186993-4914-43bf-ae43-05199ed9872e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:52.258276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06186993-4914-43bf-ae43-05199ed9872e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:52.259664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 596/1167] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:52.264755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dade8-9192-41c6-8ebd-21a9fc59f321 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:52.265157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dade8-9192-41c6-8ebd-21a9fc59f321 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:52.265319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dade8-9192-41c6-8ebd-21a9fc59f321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:52.282099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ed6dade8-9192-41c6-8ebd-21a9fc59f321 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:52.283309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ed6dade8-9192-41c6-8ebd-21a9fc59f321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:52.294277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:52.348077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 572/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:52.371243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06186993-4914-43bf-ae43-05199ed9872e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:52.472339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06186993-4914-43bf-ae43-05199ed9872e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:52.497936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:52.505881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36fe06-a3ee-4e74-880c-1d115662688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:52.506541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36fe06-a3ee-4e74-880c-1d115662688c None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:52.506830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36fe06-a3ee-4e74-880c-1d115662688c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:52.507112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36fe06-a3ee-4e74-880c-1d115662688c None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:52.507936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36fe06-a3ee-4e74-880c-1d115662688c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:52.509248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36fe06-a3ee-4e74-880c-1d115662688c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:52.523523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 597/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Fri Jul 22 11:02:52 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:02:54.551158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f847bb-27b9-4ed6-8407-4f5ad30dc76f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:54.551761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f847bb-27b9-4ed6-8407-4f5ad30dc76f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:54.551761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f847bb-27b9-4ed6-8407-4f5ad30dc76f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:54.553183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 573/1170] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:54.558240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77ef6-b288-4f47-adeb-d9119d30b3a5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:54.558648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77ef6-b288-4f47-adeb-d9119d30b3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:54.558810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77ef6-b288-4f47-adeb-d9119d30b3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:54.574960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-aef77ef6-b288-4f47-adeb-d9119d30b3a5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:54.576176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aef77ef6-b288-4f47-adeb-d9119d30b3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:54.640769 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 598/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:54.664755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f847bb-27b9-4ed6-8407-4f5ad30dc76f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:54.762703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f847bb-27b9-4ed6-8407-4f5ad30dc76f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:54.795848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79049f17-13dd-4945-926e-9e5caff50701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:54.796698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79049f17-13dd-4945-926e-9e5caff50701 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:54.796987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79049f17-13dd-4945-926e-9e5caff50701 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:54.797268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79049f17-13dd-4945-926e-9e5caff50701 None admin] PATH_INFO: `/v3/domains/1dd5b5d76eeb4a70b845913b44bbc410` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:54.807672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79049f17-13dd-4945-926e-9e5caff50701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1dd5b5d76eeb4a70b845913b44bbc410)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:54.809059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79049f17-13dd-4945-926e-9e5caff50701 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:54.810696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 574/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Fri Jul 22 11:02:54 2022] GET /identity/v3/domains/1dd5b5d76eeb4a70b845913b44bbc410 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:54.816448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef77ef6-b288-4f47-adeb-d9119d30b3a5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:54.911448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef77ef6-b288-4f47-adeb-d9119d30b3a5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:54.927721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:54.945630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7182d-1e12-4047-83df-2ded66f67bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:54.946285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7182d-1e12-4047-83df-2ded66f67bc5 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:54.946576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7182d-1e12-4047-83df-2ded66f67bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:54.946859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7182d-1e12-4047-83df-2ded66f67bc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:54.947599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7182d-1e12-4047-83df-2ded66f67bc5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:54.949021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7182d-1e12-4047-83df-2ded66f67bc5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:54.973852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 599/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:02:54 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:02:57.057391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519d929-2c4b-468c-8cd5-b65f4a3b9a96 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:57.057913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519d929-2c4b-468c-8cd5-b65f4a3b9a96 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:57.058010 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519d929-2c4b-468c-8cd5-b65f4a3b9a96 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:57.059357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 575/1174] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:57.064422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fc074-cac5-4c28-a99b-b3c94d5d5874 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:57.064854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fc074-cac5-4c28-a99b-b3c94d5d5874 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:57.064989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fc074-cac5-4c28-a99b-b3c94d5d5874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:57.081305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d77fc074-cac5-4c28-a99b-b3c94d5d5874 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:57.082466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d77fc074-cac5-4c28-a99b-b3c94d5d5874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:57.145559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 600/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:02:57.174255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519d929-2c4b-468c-8cd5-b65f4a3b9a96 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:57.184130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:57.275030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519d929-2c4b-468c-8cd5-b65f4a3b9a96 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:57.309636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f31700-9889-4b41-bbf9-916f76fe58b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:57.310420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f31700-9889-4b41-bbf9-916f76fe58b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:57.310707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f31700-9889-4b41-bbf9-916f76fe58b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:57.310983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f31700-9889-4b41-bbf9-916f76fe58b2 None admin] PATH_INFO: `/v3/domains/1dd5b5d76eeb4a70b845913b44bbc410` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:57.312177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f31700-9889-4b41-bbf9-916f76fe58b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1dd5b5d76eeb4a70b845913b44bbc410)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:57.313617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f31700-9889-4b41-bbf9-916f76fe58b2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:57.315237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 576/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Fri Jul 22 11:02:57 2022] GET /identity/v3/domains/1dd5b5d76eeb4a70b845913b44bbc410 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:57.321193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fc074-cac5-4c28-a99b-b3c94d5d5874 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:57.414400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:57.417269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fc074-cac5-4c28-a99b-b3c94d5d5874 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:57.451189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2b344-f080-49a7-accb-9c9a4baf833a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:57.451877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2b344-f080-49a7-accb-9c9a4baf833a None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:57.452164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2b344-f080-49a7-accb-9c9a4baf833a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:57.452443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2b344-f080-49a7-accb-9c9a4baf833a None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:57.453136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2b344-f080-49a7-accb-9c9a4baf833a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:57.454571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2b344-f080-49a7-accb-9c9a4baf833a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:57.474547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 601/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:02:57 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:02:59.534746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1095829-73ff-4006-adc0-49eea494f2f0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:59.535311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1095829-73ff-4006-adc0-49eea494f2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:59.535311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1095829-73ff-4006-adc0-49eea494f2f0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:59.537298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 577/1178] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:02:59.542428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89533be3-fbf7-46a6-9483-6a30242608f9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:59.542836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89533be3-fbf7-46a6-9483-6a30242608f9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:59.543001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89533be3-fbf7-46a6-9483-6a30242608f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:59.559994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-89533be3-fbf7-46a6-9483-6a30242608f9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:02:59.561196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89533be3-fbf7-46a6-9483-6a30242608f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:02:59.628985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 602/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:02:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 22 11:02:59.652266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1095829-73ff-4006-adc0-49eea494f2f0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:59.727270 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:59.752028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1095829-73ff-4006-adc0-49eea494f2f0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:59.783747 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c27a3-ad3a-4a13-bda0-fad7ff92db9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:59.784522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c27a3-ad3a-4a13-bda0-fad7ff92db9d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:59.784850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c27a3-ad3a-4a13-bda0-fad7ff92db9d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:59.785073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c27a3-ad3a-4a13-bda0-fad7ff92db9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:59.795731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c27a3-ad3a-4a13-bda0-fad7ff92db9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-263c27a3-ad3a-4a13-bda0-fad7ff92db9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:02:59.803137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:02:59.804326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:02:59.805592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 578/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:02:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:02:59.810310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89533be3-fbf7-46a6-9483-6a30242608f9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:02:59.896425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:02:59.913963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89533be3-fbf7-46a6-9483-6a30242608f9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:02:59.947024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46955a-5c7e-41a6-b142-4963e34d7980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:02:59.947723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46955a-5c7e-41a6-b142-4963e34d7980 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:02:59.948014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46955a-5c7e-41a6-b142-4963e34d7980 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:02:59.948297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46955a-5c7e-41a6-b142-4963e34d7980 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:02:59.949045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46955a-5c7e-41a6-b142-4963e34d7980 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:02:59.949356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46955a-5c7e-41a6-b142-4963e34d7980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:02:59.950725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46955a-5c7e-41a6-b142-4963e34d7980 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:02:59.959145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 603/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:02:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:02:59.964990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c27a3-ad3a-4a13-bda0-fad7ff92db9d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:00.060783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c27a3-ad3a-4a13-bda0-fad7ff92db9d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:00.093943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495efbae-f4f0-4a42-aa5d-beab9d904c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:00.094803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495efbae-f4f0-4a42-aa5d-beab9d904c4a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:00.095088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495efbae-f4f0-4a42-aa5d-beab9d904c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:00.095398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495efbae-f4f0-4a42-aa5d-beab9d904c4a None admin] PATH_INFO: `/v3/users/f674734581664d30b3258547a12776c3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:00.097284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:00.109432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495efbae-f4f0-4a42-aa5d-beab9d904c4a None admin] RBAC: Authorizing `identity:get_user(user_id=f674734581664d30b3258547a12776c3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:00.110872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495efbae-f4f0-4a42-aa5d-beab9d904c4a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:00.112815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 579/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:02:59 2022] GET /identity/v3/users/f674734581664d30b3258547a12776c3 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:00.118664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46955a-5c7e-41a6-b142-4963e34d7980 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:00.218837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46955a-5c7e-41a6-b142-4963e34d7980 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:00.256325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abe71a-2757-4dfc-8369-f657d0d5c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:00.256969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abe71a-2757-4dfc-8369-f657d0d5c819 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:00.257251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abe71a-2757-4dfc-8369-f657d0d5c819 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:00.257529 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abe71a-2757-4dfc-8369-f657d0d5c819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:00.275030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:00.322566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abe71a-2757-4dfc-8369-f657d0d5c819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:00.323977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abe71a-2757-4dfc-8369-f657d0d5c819 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:00.417540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 604/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:03:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:03:00.423703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495efbae-f4f0-4a42-aa5d-beab9d904c4a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:00.523515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495efbae-f4f0-4a42-aa5d-beab9d904c4a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:00.555638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0647694-53e7-461e-9829-82288366e51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:00.556480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0647694-53e7-461e-9829-82288366e51e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:00.556761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0647694-53e7-461e-9829-82288366e51e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:00.557031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0647694-53e7-461e-9829-82288366e51e None admin] PATH_INFO: `/v3/projects/c40862d448fc4979b71183c87bb6b68e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:00.558184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0647694-53e7-461e-9829-82288366e51e None admin] RBAC: Authorizing `identity:get_project(project_id=c40862d448fc4979b71183c87bb6b68e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:00.559625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0647694-53e7-461e-9829-82288366e51e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:00.561297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 580/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:03:00 2022] GET /identity/v3/projects/c40862d448fc4979b71183c87bb6b68e => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:00.567447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abe71a-2757-4dfc-8369-f657d0d5c819 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:00.615728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:00.664980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abe71a-2757-4dfc-8369-f657d0d5c819 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:00.699868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eec8fa-8cb3-405e-88e9-219d966b8f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:00.700515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eec8fa-8cb3-405e-88e9-219d966b8f4b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:00.700799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eec8fa-8cb3-405e-88e9-219d966b8f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:00.701078 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eec8fa-8cb3-405e-88e9-219d966b8f4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:00.701899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eec8fa-8cb3-405e-88e9-219d966b8f4b None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=c40862d448fc4979b71183c87bb6b68e, user.id=f674734581664d30b3258547a12776c3) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:03:00.702209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eec8fa-8cb3-405e-88e9-219d966b8f4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:00.703659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eec8fa-8cb3-405e-88e9-219d966b8f4b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:00.709324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 605/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:03:00 2022] GET /identity/v3/role_assignments?user.id=f674734581664d30b3258547a12776c3&scope.project.id=c40862d448fc4979b71183c87bb6b68e&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:02.705321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167e52b-ac9c-43c2-a42a-b7714e1d1ca7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:02.705847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167e52b-ac9c-43c2-a42a-b7714e1d1ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:02.705942 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167e52b-ac9c-43c2-a42a-b7714e1d1ca7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:02.707247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 581/1186] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:03:02.713100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efd2db-beff-4ed1-b906-915d13eefb3c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:02.713504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efd2db-beff-4ed1-b906-915d13eefb3c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:02.713644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efd2db-beff-4ed1-b906-915d13eefb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:02.730778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-22efd2db-beff-4ed1-b906-915d13eefb3c None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:03:02.732006 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-22efd2db-beff-4ed1-b906-915d13eefb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:03:02.796453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 606/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:03:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:03:02.818379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167e52b-ac9c-43c2-a42a-b7714e1d1ca7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:02.885905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:02.926129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167e52b-ac9c-43c2-a42a-b7714e1d1ca7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:02.957775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adb4ac-7d12-40a1-8efd-4e6cd508a0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:02.959065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adb4ac-7d12-40a1-8efd-4e6cd508a0ac None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:02.959382 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adb4ac-7d12-40a1-8efd-4e6cd508a0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:02.959662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adb4ac-7d12-40a1-8efd-4e6cd508a0ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:02.969363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adb4ac-7d12-40a1-8efd-4e6cd508a0ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-91adb4ac-7d12-40a1-8efd-4e6cd508a0ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:03:02.977548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:03:02.978643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:03:02.979620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 582/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:03:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:03:02.984726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efd2db-beff-4ed1-b906-915d13eefb3c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:03.026134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:03.080408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efd2db-beff-4ed1-b906-915d13eefb3c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:03.113108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da046bdc-563b-48be-b740-4c0ea352be33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:03.113771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046bdc-563b-48be-b740-4c0ea352be33 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:03.114063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046bdc-563b-48be-b740-4c0ea352be33 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:03.114345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046bdc-563b-48be-b740-4c0ea352be33 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:03.115088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da046bdc-563b-48be-b740-4c0ea352be33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:03:03.115455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da046bdc-563b-48be-b740-4c0ea352be33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:03.116867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da046bdc-563b-48be-b740-4c0ea352be33 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:03.124675 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 607/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:03:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:03.130996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adb4ac-7d12-40a1-8efd-4e6cd508a0ac None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:03.229044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adb4ac-7d12-40a1-8efd-4e6cd508a0ac None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:03.247743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:03.261700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69027516-6640-4081-b0ee-3c50f1e87362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:03.262555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69027516-6640-4081-b0ee-3c50f1e87362 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:03.262839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69027516-6640-4081-b0ee-3c50f1e87362 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:03.263113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69027516-6640-4081-b0ee-3c50f1e87362 None admin] PATH_INFO: `/v3/users/f674734581664d30b3258547a12776c3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:03.264308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69027516-6640-4081-b0ee-3c50f1e87362 None admin] RBAC: Authorizing `identity:get_user(user_id=f674734581664d30b3258547a12776c3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:03.265739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69027516-6640-4081-b0ee-3c50f1e87362 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:03.267517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 583/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:03:03 2022] GET /identity/v3/users/f674734581664d30b3258547a12776c3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:03.273647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da046bdc-563b-48be-b740-4c0ea352be33 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:03.368843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da046bdc-563b-48be-b740-4c0ea352be33 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:03.371071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:03.401966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba72cb-051c-4f1a-afbe-d3e83c1bc3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:03.402823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba72cb-051c-4f1a-afbe-d3e83c1bc3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:03.403112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba72cb-051c-4f1a-afbe-d3e83c1bc3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:03.403425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba72cb-051c-4f1a-afbe-d3e83c1bc3b2 None admin] PATH_INFO: `/v3/projects/c40862d448fc4979b71183c87bb6b68e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:03.404589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba72cb-051c-4f1a-afbe-d3e83c1bc3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=c40862d448fc4979b71183c87bb6b68e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:03.406012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba72cb-051c-4f1a-afbe-d3e83c1bc3b2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:03.407742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 608/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:03:03 2022] GET /identity/v3/projects/c40862d448fc4979b71183c87bb6b68e => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:03.413628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69027516-6640-4081-b0ee-3c50f1e87362 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:03.510881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69027516-6640-4081-b0ee-3c50f1e87362 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:03.542334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fe669-65ad-4f8f-8927-63c0c71d47ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:03.543200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fe669-65ad-4f8f-8927-63c0c71d47ef None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:03.543512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fe669-65ad-4f8f-8927-63c0c71d47ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:03.543787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fe669-65ad-4f8f-8927-63c0c71d47ef None admin] PATH_INFO: `/v3/projects/c40862d448fc4979b71183c87bb6b68e/users/f674734581664d30b3258547a12776c3/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:03.545714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fe669-65ad-4f8f-8927-63c0c71d47ef None admin] RBAC: Authorizing `identity:create_grant(project_id=c40862d448fc4979b71183c87bb6b68e, user_id=f674734581664d30b3258547a12776c3, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:03.547133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fe669-65ad-4f8f-8927-63c0c71d47ef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:03.560032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 584/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:03:03 2022] PUT /identity/v3/projects/c40862d448fc4979b71183c87bb6b68e/users/f674734581664d30b3258547a12776c3/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:03:05.552244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568554e0-1e2a-45cf-9320-916a249736a2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:05.552799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568554e0-1e2a-45cf-9320-916a249736a2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:05.552799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568554e0-1e2a-45cf-9320-916a249736a2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:05.554169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 609/1193] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:03:05.559421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1652-65b1-42c6-8da3-3029014cd8a8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:05.559847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1652-65b1-42c6-8da3-3029014cd8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:05.559979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1652-65b1-42c6-8da3-3029014cd8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:05.576779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c1ec1652-65b1-42c6-8da3-3029014cd8a8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:03:05.577945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1ec1652-65b1-42c6-8da3-3029014cd8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:03:05.620559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:05.643830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 585/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:03:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:03:05.667057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568554e0-1e2a-45cf-9320-916a249736a2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:05.765660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568554e0-1e2a-45cf-9320-916a249736a2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:05.797494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd3c58-7de0-4879-8637-c5db0f0bb7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:05.798278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd3c58-7de0-4879-8637-c5db0f0bb7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:05.798566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd3c58-7de0-4879-8637-c5db0f0bb7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:05.798847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd3c58-7de0-4879-8637-c5db0f0bb7e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:05.809189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd3c58-7de0-4879-8637-c5db0f0bb7e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b8cd3c58-7de0-4879-8637-c5db0f0bb7e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:03:05.816673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:03:05.818295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:03:05.819399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 610/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:03:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:03:05.824416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec1652-65b1-42c6-8da3-3029014cd8a8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:05.917787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec1652-65b1-42c6-8da3-3029014cd8a8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:05.950841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec4ecf-86bb-48f6-b3a4-6d6932eb8f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:05.951520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec4ecf-86bb-48f6-b3a4-6d6932eb8f97 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:05.951804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec4ecf-86bb-48f6-b3a4-6d6932eb8f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:05.952079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec4ecf-86bb-48f6-b3a4-6d6932eb8f97 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:05.952817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec4ecf-86bb-48f6-b3a4-6d6932eb8f97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:03:05.953122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec4ecf-86bb-48f6-b3a4-6d6932eb8f97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:05.954475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec4ecf-86bb-48f6-b3a4-6d6932eb8f97 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:05.962555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 586/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:03:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:05.968081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd3c58-7de0-4879-8637-c5db0f0bb7e2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:05.999131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:06.066444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd3c58-7de0-4879-8637-c5db0f0bb7e2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:06.099764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453c53-53b5-4434-8720-01c4971b04fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:06.100623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453c53-53b5-4434-8720-01c4971b04fb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:06.100919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453c53-53b5-4434-8720-01c4971b04fb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:06.101187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453c53-53b5-4434-8720-01c4971b04fb None admin] PATH_INFO: `/v3/users/f674734581664d30b3258547a12776c3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:06.102355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453c53-53b5-4434-8720-01c4971b04fb None admin] RBAC: Authorizing `identity:get_user(user_id=f674734581664d30b3258547a12776c3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:06.103829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453c53-53b5-4434-8720-01c4971b04fb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:06.105439 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 611/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Jul 22 11:03:05 2022] GET /identity/v3/users/f674734581664d30b3258547a12776c3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:06.111012 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec4ecf-86bb-48f6-b3a4-6d6932eb8f97 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:06.152355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:06.208317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec4ecf-86bb-48f6-b3a4-6d6932eb8f97 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:06.241052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56e8f5-b5fe-4cce-93a8-1da48bb60ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:06.241683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56e8f5-b5fe-4cce-93a8-1da48bb60ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:06.241968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56e8f5-b5fe-4cce-93a8-1da48bb60ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:06.242246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56e8f5-b5fe-4cce-93a8-1da48bb60ec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:06.307019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56e8f5-b5fe-4cce-93a8-1da48bb60ec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:06.308490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56e8f5-b5fe-4cce-93a8-1da48bb60ec8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:06.345551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:06.401558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 587/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:03:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:03:06.407432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453c53-53b5-4434-8720-01c4971b04fb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:06.504341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:06.507604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453c53-53b5-4434-8720-01c4971b04fb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:06.542141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24acf3-b2a5-45c4-b488-8696479a4114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:06.542672 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24acf3-b2a5-45c4-b488-8696479a4114 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:06.542963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24acf3-b2a5-45c4-b488-8696479a4114 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:06.543246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24acf3-b2a5-45c4-b488-8696479a4114 None admin] PATH_INFO: `/v3/projects/c40862d448fc4979b71183c87bb6b68e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:06.544475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24acf3-b2a5-45c4-b488-8696479a4114 None admin] RBAC: Authorizing `identity:get_project(project_id=c40862d448fc4979b71183c87bb6b68e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:06.545920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24acf3-b2a5-45c4-b488-8696479a4114 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:06.547674 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 612/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Jul 22 11:03:06 2022] GET /identity/v3/projects/c40862d448fc4979b71183c87bb6b68e => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:06.553777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56e8f5-b5fe-4cce-93a8-1da48bb60ec8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:03:06.650209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56e8f5-b5fe-4cce-93a8-1da48bb60ec8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:03:06.682613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb75064-05c0-46d4-b8d0-f9a40de5cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:03:06.683283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb75064-05c0-46d4-b8d0-f9a40de5cc81 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:03:06.683568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb75064-05c0-46d4-b8d0-f9a40de5cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:03:06.683840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb75064-05c0-46d4-b8d0-f9a40de5cc81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:03:06.684659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb75064-05c0-46d4-b8d0-f9a40de5cc81 None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=c40862d448fc4979b71183c87bb6b68e, user.id=f674734581664d30b3258547a12776c3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:03:06.684968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb75064-05c0-46d4-b8d0-f9a40de5cc81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:03:06.686393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb75064-05c0-46d4-b8d0-f9a40de5cc81 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:03:06.692039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 588/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:03:06 2022] GET /identity/v3/role_assignments?user.id=f674734581664d30b3258547a12776c3&scope.project.id=c40862d448fc4979b71183c87bb6b68e&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:03:16.537665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=16 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:03:16.689410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:01.898177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0de15-7aa9-47c8-85ad-203a8908cae3 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:01.898177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0de15-7aa9-47c8-85ad-203a8908cae3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:01.898177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0de15-7aa9-47c8-85ad-203a8908cae3 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:01.899377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 613/1201] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:04:01.904784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1d113-d603-40de-ab68-eacb8e94fee6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:01.905586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1d113-d603-40de-ab68-eacb8e94fee6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:01.905907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1d113-d603-40de-ab68-eacb8e94fee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:01.927412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b0d1d113-d603-40de-ab68-eacb8e94fee6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:01.930633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b0d1d113-d603-40de-ab68-eacb8e94fee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:02.027598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 589/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:02.056558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0de15-7aa9-47c8-85ad-203a8908cae3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:02.209233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0de15-7aa9-47c8-85ad-203a8908cae3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:02.259721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67228f-6c72-4509-8a22-d3a0eed56aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:02.260915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67228f-6c72-4509-8a22-d3a0eed56aae None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:02.261216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67228f-6c72-4509-8a22-d3a0eed56aae None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:02.261503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67228f-6c72-4509-8a22-d3a0eed56aae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:02.275228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67228f-6c72-4509-8a22-d3a0eed56aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:02.276923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67228f-6c72-4509-8a22-d3a0eed56aae None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4e67228f-6c72-4509-8a22-d3a0eed56aae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:02.298134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:02.299635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:02.300685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:02.300685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:02.300685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:02.300685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:02.300685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 614/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:02.305826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1d113-d603-40de-ab68-eacb8e94fee6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:02.431997 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1d113-d603-40de-ab68-eacb8e94fee6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:02.470380 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:02.477633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0401f2d-ef87-442a-a58c-445b9e4e8a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:02.478310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0401f2d-ef87-442a-a58c-445b9e4e8a72 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:02.478605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0401f2d-ef87-442a-a58c-445b9e4e8a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:02.478890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0401f2d-ef87-442a-a58c-445b9e4e8a72 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:02.479654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0401f2d-ef87-442a-a58c-445b9e4e8a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:02.479974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0401f2d-ef87-442a-a58c-445b9e4e8a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:02.481395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0401f2d-ef87-442a-a58c-445b9e4e8a72 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:02.498352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 590/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:02.504566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67228f-6c72-4509-8a22-d3a0eed56aae None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:02.577612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:02.650708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67228f-6c72-4509-8a22-d3a0eed56aae None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:02.701310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c754cc8-afb6-4f4f-ab3b-c0135d37774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:02.702075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c754cc8-afb6-4f4f-ab3b-c0135d37774c None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:02.702398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c754cc8-afb6-4f4f-ab3b-c0135d37774c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:02.702717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c754cc8-afb6-4f4f-ab3b-c0135d37774c None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:02.703506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c754cc8-afb6-4f4f-ab3b-c0135d37774c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:02.704946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c754cc8-afb6-4f4f-ab3b-c0135d37774c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:02.728743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 615/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Jul 22 11:04:02 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:04:05.006805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01447d-7f01-4e88-8411-011ac7f79796 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:05.007337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01447d-7f01-4e88-8411-011ac7f79796 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:05.007337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01447d-7f01-4e88-8411-011ac7f79796 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:05.008976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 591/1206] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:05.014715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd4ae4-8340-4cb7-8631-c2cd708ffa4a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:05.015190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd4ae4-8340-4cb7-8631-c2cd708ffa4a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:05.015190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd4ae4-8340-4cb7-8631-c2cd708ffa4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:05.034290 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-eedd4ae4-8340-4cb7-8631-c2cd708ffa4a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:05.035623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eedd4ae4-8340-4cb7-8631-c2cd708ffa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:05.113792 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 616/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:05.138732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01447d-7f01-4e88-8411-011ac7f79796 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:05.276536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01447d-7f01-4e88-8411-011ac7f79796 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:05.315047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05165545-778d-4b38-9d77-aa0800ebf738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:05.315894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05165545-778d-4b38-9d77-aa0800ebf738 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:05.316264 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05165545-778d-4b38-9d77-aa0800ebf738 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:05.316505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05165545-778d-4b38-9d77-aa0800ebf738 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:05.329721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05165545-778d-4b38-9d77-aa0800ebf738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-05165545-778d-4b38-9d77-aa0800ebf738 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:04:05.339083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:05.340218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:04:05.341083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:05.346264 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 592/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:04:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:05.348084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd4ae4-8340-4cb7-8631-c2cd708ffa4a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:05.392483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:05.466515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd4ae4-8340-4cb7-8631-c2cd708ffa4a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:05.505141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc4524-2249-4552-aa38-73698fcc6a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:05.505957 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc4524-2249-4552-aa38-73698fcc6a2f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:05.506318 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc4524-2249-4552-aa38-73698fcc6a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:05.506644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc4524-2249-4552-aa38-73698fcc6a2f None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:05.507497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc4524-2249-4552-aa38-73698fcc6a2f None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:05.507917 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc4524-2249-4552-aa38-73698fcc6a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:05.509594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc4524-2249-4552-aa38-73698fcc6a2f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:05.519824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 617/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:04:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:04:05.525981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05165545-778d-4b38-9d77-aa0800ebf738 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:05.588283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:05.644222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05165545-778d-4b38-9d77-aa0800ebf738 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:05.690263 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009f606-572c-4a67-8f4b-36671d784171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:05.691139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009f606-572c-4a67-8f4b-36671d784171 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:05.691504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009f606-572c-4a67-8f4b-36671d784171 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:05.691808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009f606-572c-4a67-8f4b-36671d784171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:05.703368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009f606-572c-4a67-8f4b-36671d784171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:05.704791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009f606-572c-4a67-8f4b-36671d784171 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9009f606-572c-4a67-8f4b-36671d784171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:05.724297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:05.725255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:05.726189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:05.726189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:05.726189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:05.726189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:05.726189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 593/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:05.731735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc4524-2249-4552-aa38-73698fcc6a2f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:05.838839 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:05.842158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc4524-2249-4552-aa38-73698fcc6a2f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:05.885384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df708c4-6441-4a3e-b7ac-bed6717ce4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:05.886112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df708c4-6441-4a3e-b7ac-bed6717ce4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:05.886469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df708c4-6441-4a3e-b7ac-bed6717ce4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:05.886776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df708c4-6441-4a3e-b7ac-bed6717ce4b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:05.887523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df708c4-6441-4a3e-b7ac-bed6717ce4b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:05.887853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df708c4-6441-4a3e-b7ac-bed6717ce4b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:05.889334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df708c4-6441-4a3e-b7ac-bed6717ce4b1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:05.902933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 618/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:05.908996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009f606-572c-4a67-8f4b-36671d784171 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:06.020267 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009f606-572c-4a67-8f4b-36671d784171 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:06.022189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:06.058874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed105090-3204-41bb-8727-2051b4d5c6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:06.059753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105090-3204-41bb-8727-2051b4d5c6da None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:06.060149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105090-3204-41bb-8727-2051b4d5c6da None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:06.060467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105090-3204-41bb-8727-2051b4d5c6da None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:06.068468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed105090-3204-41bb-8727-2051b4d5c6da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:06.070011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed105090-3204-41bb-8727-2051b4d5c6da None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ed105090-3204-41bb-8727-2051b4d5c6da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:04:06.085733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:06.086767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:06.087837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 594/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:06.093063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df708c4-6441-4a3e-b7ac-bed6717ce4b1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:06.207139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df708c4-6441-4a3e-b7ac-bed6717ce4b1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:06.246901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6bbf2-6d07-4745-98bd-bb4cf2f7f63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:06.247636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6bbf2-6d07-4745-98bd-bb4cf2f7f63f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:06.247958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6bbf2-6d07-4745-98bd-bb4cf2f7f63f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:06.248263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6bbf2-6d07-4745-98bd-bb4cf2f7f63f None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:06.249082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6bbf2-6d07-4745-98bd-bb4cf2f7f63f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:06.249412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6bbf2-6d07-4745-98bd-bb4cf2f7f63f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:06.250922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6bbf2-6d07-4745-98bd-bb4cf2f7f63f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:06.274859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 619/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 22 11:04:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:06.276112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed105090-3204-41bb-8727-2051b4d5c6da None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:06.395099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed105090-3204-41bb-8727-2051b4d5c6da None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:06.436576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa92f81-234f-460d-bc7a-437d43682b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:06.437657 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa92f81-234f-460d-bc7a-437d43682b1b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:06.438158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa92f81-234f-460d-bc7a-437d43682b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:06.438635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa92f81-234f-460d-bc7a-437d43682b1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:06.474249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:06.518580 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa92f81-234f-460d-bc7a-437d43682b1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:06.520168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa92f81-234f-460d-bc7a-437d43682b1b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:06.637881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 595/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:04:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 22 11:04:06.644342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6bbf2-6d07-4745-98bd-bb4cf2f7f63f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:06.666396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:06.765017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6bbf2-6d07-4745-98bd-bb4cf2f7f63f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:06.805756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dbce4a-d838-4b2f-ab5b-1f139e8bcb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:06.806560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dbce4a-d838-4b2f-ab5b-1f139e8bcb73 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:06.806851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dbce4a-d838-4b2f-ab5b-1f139e8bcb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:06.807133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dbce4a-d838-4b2f-ab5b-1f139e8bcb73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:06.819613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dbce4a-d838-4b2f-ab5b-1f139e8bcb73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:06.821144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dbce4a-d838-4b2f-ab5b-1f139e8bcb73 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-58dbce4a-d838-4b2f-ab5b-1f139e8bcb73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:06.841008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:06.842175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:06.843314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:06.843314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:06.843314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:06.843314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:06.847405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 620/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:06.848524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa92f81-234f-460d-bc7a-437d43682b1b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:06.924086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:06.971239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa92f81-234f-460d-bc7a-437d43682b1b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:07.012155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4cf2f-714d-4cda-8364-952b83cba272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:07.012756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4cf2f-714d-4cda-8364-952b83cba272 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:07.013080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4cf2f-714d-4cda-8364-952b83cba272 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:07.013494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4cf2f-714d-4cda-8364-952b83cba272 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:07.014228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4cf2f-714d-4cda-8364-952b83cba272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:07.014552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4cf2f-714d-4cda-8364-952b83cba272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:07.016109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4cf2f-714d-4cda-8364-952b83cba272 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:07.033110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 596/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:04:07.034384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dbce4a-d838-4b2f-ab5b-1f139e8bcb73 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:07.113648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:07.151433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dbce4a-d838-4b2f-ab5b-1f139e8bcb73 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:07.196675 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67058f41-478d-401a-8e23-a86fb72ccdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:07.197407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058f41-478d-401a-8e23-a86fb72ccdfd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:07.197697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058f41-478d-401a-8e23-a86fb72ccdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:07.197975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058f41-478d-401a-8e23-a86fb72ccdfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:07.205602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058f41-478d-401a-8e23-a86fb72ccdfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:07.207487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058f41-478d-401a-8e23-a86fb72ccdfd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-67058f41-478d-401a-8e23-a86fb72ccdfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:04:07.223016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:07.224005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:07.224969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:04:07.224969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:04:07.224969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:04:07.224969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:04:07.224969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:04:07.224969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:04:07.224969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:07.224969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:07.227498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 621/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:04:07.230496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4cf2f-714d-4cda-8364-952b83cba272 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:07.345567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4cf2f-714d-4cda-8364-952b83cba272 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:07.374021 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:07.388617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423056ef-063a-4eab-9ed0-5d8fbe5cfe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:07.389289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423056ef-063a-4eab-9ed0-5d8fbe5cfe95 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:07.389571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423056ef-063a-4eab-9ed0-5d8fbe5cfe95 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:07.389850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423056ef-063a-4eab-9ed0-5d8fbe5cfe95 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:07.390640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423056ef-063a-4eab-9ed0-5d8fbe5cfe95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:07.390951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423056ef-063a-4eab-9ed0-5d8fbe5cfe95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:07.392412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423056ef-063a-4eab-9ed0-5d8fbe5cfe95 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:07.407434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 597/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:04:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:07.413540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67058f41-478d-401a-8e23-a86fb72ccdfd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:07.534058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67058f41-478d-401a-8e23-a86fb72ccdfd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:07.568281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:07.575303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ed44c-d7c7-4705-810a-5f10ff5f7c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:07.575303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ed44c-d7c7-4705-810a-5f10ff5f7c5b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:07.575577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ed44c-d7c7-4705-810a-5f10ff5f7c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:07.575916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ed44c-d7c7-4705-810a-5f10ff5f7c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:07.576688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ed44c-d7c7-4705-810a-5f10ff5f7c5b None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=88ecd4c645174d28930a44b93cfee9b6) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:07.577192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ed44c-d7c7-4705-810a-5f10ff5f7c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:07.581145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ed44c-d7c7-4705-810a-5f10ff5f7c5b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:07.584919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 622/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:04:07 2022] GET /identity/v3/role_assignments?user.id=88ecd4c645174d28930a44b93cfee9b6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:09.588006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa0639-47e2-43da-b16d-94e54c8b9742 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:09.588591 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa0639-47e2-43da-b16d-94e54c8b9742 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:09.588591 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa0639-47e2-43da-b16d-94e54c8b9742 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:09.590108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 598/1220] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:09.597080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c19ad-bf69-4605-9b32-ac1627a16028 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:09.597601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c19ad-bf69-4605-9b32-ac1627a16028 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:09.598179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c19ad-bf69-4605-9b32-ac1627a16028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:09.615781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-626c19ad-bf69-4605-9b32-ac1627a16028 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:09.616978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-626c19ad-bf69-4605-9b32-ac1627a16028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:09.684608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 623/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:09.707406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa0639-47e2-43da-b16d-94e54c8b9742 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:09.822090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa0639-47e2-43da-b16d-94e54c8b9742 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:09.860617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20324d9f-82cd-48c6-a08e-49fb86b19c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:09.862187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20324d9f-82cd-48c6-a08e-49fb86b19c87 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:09.862680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20324d9f-82cd-48c6-a08e-49fb86b19c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:09.863193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20324d9f-82cd-48c6-a08e-49fb86b19c87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:09.876068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20324d9f-82cd-48c6-a08e-49fb86b19c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-20324d9f-82cd-48c6-a08e-49fb86b19c87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:04:09.886336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:09.887426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:09.888290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 599/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:04:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:09.894553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c19ad-bf69-4605-9b32-ac1627a16028 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:10.014539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c19ad-bf69-4605-9b32-ac1627a16028 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:10.044312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:10.058193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6933b-fca8-4535-a24f-5e406af4c5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:10.059012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6933b-fca8-4535-a24f-5e406af4c5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:10.059370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6933b-fca8-4535-a24f-5e406af4c5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:10.059680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6933b-fca8-4535-a24f-5e406af4c5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:10.060435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6933b-fca8-4535-a24f-5e406af4c5f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:10.060746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6933b-fca8-4535-a24f-5e406af4c5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:10.062362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6933b-fca8-4535-a24f-5e406af4c5f3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:10.072753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 624/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:04:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:10.078590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20324d9f-82cd-48c6-a08e-49fb86b19c87 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:10.127635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:10.197961 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20324d9f-82cd-48c6-a08e-49fb86b19c87 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:10.234511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409e682-55d8-475d-a5c1-75b45d5e3df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:10.235473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409e682-55d8-475d-a5c1-75b45d5e3df9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:10.235868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409e682-55d8-475d-a5c1-75b45d5e3df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:10.237011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409e682-55d8-475d-a5c1-75b45d5e3df9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:10.249169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409e682-55d8-475d-a5c1-75b45d5e3df9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:10.250700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409e682-55d8-475d-a5c1-75b45d5e3df9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a409e682-55d8-475d-a5c1-75b45d5e3df9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:10.270669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:10.272005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:10.272937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:10.272937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:10.272937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:10.272937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:10.272937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 600/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:10.278149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6933b-fca8-4535-a24f-5e406af4c5f3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:10.385167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6933b-fca8-4535-a24f-5e406af4c5f3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:10.422023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b0d1f-820e-4a28-870c-b140049635f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:10.422788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b0d1f-820e-4a28-870c-b140049635f9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:10.423150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b0d1f-820e-4a28-870c-b140049635f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:10.423528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b0d1f-820e-4a28-870c-b140049635f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:10.425031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b0d1f-820e-4a28-870c-b140049635f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:10.425420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b0d1f-820e-4a28-870c-b140049635f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:10.426859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b0d1f-820e-4a28-870c-b140049635f9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:10.439344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 625/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:10.445806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409e682-55d8-475d-a5c1-75b45d5e3df9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:10.555556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:10.563942 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409e682-55d8-475d-a5c1-75b45d5e3df9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:10.607183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4ed09-2736-4396-85eb-211d3a6415a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:10.608131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ed09-2736-4396-85eb-211d3a6415a3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:10.608490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ed09-2736-4396-85eb-211d3a6415a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:10.608860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ed09-2736-4396-85eb-211d3a6415a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:10.617898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4ed09-2736-4396-85eb-211d3a6415a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:10.619366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4ed09-2736-4396-85eb-211d3a6415a3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-75e4ed09-2736-4396-85eb-211d3a6415a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:04:10.633721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:10.634782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:10.635821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 601/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:10.640734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b0d1f-820e-4a28-870c-b140049635f9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:10.693502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:10.752402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b0d1f-820e-4a28-870c-b140049635f9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:10.792205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1482-b7fc-4243-af7a-5c0c3c0bddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:10.793020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1482-b7fc-4243-af7a-5c0c3c0bddf4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:10.793315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1482-b7fc-4243-af7a-5c0c3c0bddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:10.793689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1482-b7fc-4243-af7a-5c0c3c0bddf4 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:10.794758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf1482-b7fc-4243-af7a-5c0c3c0bddf4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:10.795246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf1482-b7fc-4243-af7a-5c0c3c0bddf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:10.797009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf1482-b7fc-4243-af7a-5c0c3c0bddf4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:10.823419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 626/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 22 11:04:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:10.824595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4ed09-2736-4396-85eb-211d3a6415a3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:10.928451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4ed09-2736-4396-85eb-211d3a6415a3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:10.965604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1d515-05d7-45bd-8fcf-f41db7e7ad92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:10.966627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1d515-05d7-45bd-8fcf-f41db7e7ad92 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:10.966971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1d515-05d7-45bd-8fcf-f41db7e7ad92 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:10.967308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1d515-05d7-45bd-8fcf-f41db7e7ad92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:10.980348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1d515-05d7-45bd-8fcf-f41db7e7ad92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:10.982004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1d515-05d7-45bd-8fcf-f41db7e7ad92 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-acf1d515-05d7-45bd-8fcf-f41db7e7ad92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:11.003148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:11.004431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:11.005391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:11.005391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:11.005391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:11.005391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:11.009572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 602/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:04:11.010890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1482-b7fc-4243-af7a-5c0c3c0bddf4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:11.118277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1482-b7fc-4243-af7a-5c0c3c0bddf4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:11.120928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:11.156192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a1a81-d0b3-41a4-bef8-4495f66510b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:11.156946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a1a81-d0b3-41a4-bef8-4495f66510b4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:11.157240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a1a81-d0b3-41a4-bef8-4495f66510b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:11.157556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a1a81-d0b3-41a4-bef8-4495f66510b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:11.159022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a1a81-d0b3-41a4-bef8-4495f66510b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:11.159381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a1a81-d0b3-41a4-bef8-4495f66510b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:11.160975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a1a81-d0b3-41a4-bef8-4495f66510b4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:11.175212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 627/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:11.182186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1d515-05d7-45bd-8fcf-f41db7e7ad92 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:11.194148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:11.298825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1d515-05d7-45bd-8fcf-f41db7e7ad92 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:11.336855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbccae-9ec2-4b27-8258-36b2419e2c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:11.338033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbccae-9ec2-4b27-8258-36b2419e2c5e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:11.338321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbccae-9ec2-4b27-8258-36b2419e2c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:11.338698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbccae-9ec2-4b27-8258-36b2419e2c5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:11.347463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbccae-9ec2-4b27-8258-36b2419e2c5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:11.349210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbccae-9ec2-4b27-8258-36b2419e2c5e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-50dbccae-9ec2-4b27-8258-36b2419e2c5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:04:11.365245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:11.366413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:11.367663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:04:11.367663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:04:11.367663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:04:11.367663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:04:11.367663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:04:11.367663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:04:11.367663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:11.367663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:11.367663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 603/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:04:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:11.372744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a1a81-d0b3-41a4-bef8-4495f66510b4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:11.480897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a1a81-d0b3-41a4-bef8-4495f66510b4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:11.518148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3d53b-c132-4dbb-9a16-8c549e778731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:11.518812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3d53b-c132-4dbb-9a16-8c549e778731 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:11.519213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3d53b-c132-4dbb-9a16-8c549e778731 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:11.519524 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3d53b-c132-4dbb-9a16-8c549e778731 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:11.520309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3d53b-c132-4dbb-9a16-8c549e778731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:11.520618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3d53b-c132-4dbb-9a16-8c549e778731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:11.522032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3d53b-c132-4dbb-9a16-8c549e778731 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:11.535209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 628/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:04:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:11.541186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbccae-9ec2-4b27-8258-36b2419e2c5e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:11.613050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:11.648078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbccae-9ec2-4b27-8258-36b2419e2c5e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:11.685889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e3b48-5418-4c25-978e-2dd1b49c97ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:11.686780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3b48-5418-4c25-978e-2dd1b49c97ed None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:11.687057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3b48-5418-4c25-978e-2dd1b49c97ed None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:11.687361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3b48-5418-4c25-978e-2dd1b49c97ed None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/88ecd4c645174d28930a44b93cfee9b6/roles/a9580499eef24770867aeac1602aa002` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:11.702435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e3b48-5418-4c25-978e-2dd1b49c97ed None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=88ecd4c645174d28930a44b93cfee9b6, role_id=a9580499eef24770867aeac1602aa002)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:11.703874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e3b48-5418-4c25-978e-2dd1b49c97ed None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:11.716849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 604/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:04:11 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/88ecd4c645174d28930a44b93cfee9b6/roles/a9580499eef24770867aeac1602aa002 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:04:13.716162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31696e75-74ce-49a4-a26f-b04e7373534d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:13.716703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31696e75-74ce-49a4-a26f-b04e7373534d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:13.716703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31696e75-74ce-49a4-a26f-b04e7373534d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:13.718099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 629/1233] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:13.723878 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a337a5-4f69-42fd-b6ef-51958cb21c51 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:13.724422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a337a5-4f69-42fd-b6ef-51958cb21c51 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:13.724422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a337a5-4f69-42fd-b6ef-51958cb21c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:13.741028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-06a337a5-4f69-42fd-b6ef-51958cb21c51 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:13.742221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-06a337a5-4f69-42fd-b6ef-51958cb21c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:13.813692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 605/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:13.839699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31696e75-74ce-49a4-a26f-b04e7373534d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:13.867816 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:13.955817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31696e75-74ce-49a4-a26f-b04e7373534d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:13.994846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e4d6-5e41-4ee3-9564-2ab6c410c66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:13.995692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e4d6-5e41-4ee3-9564-2ab6c410c66d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:13.995981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e4d6-5e41-4ee3-9564-2ab6c410c66d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:13.996257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e4d6-5e41-4ee3-9564-2ab6c410c66d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:14.008082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773e4d6-5e41-4ee3-9564-2ab6c410c66d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-7773e4d6-5e41-4ee3-9564-2ab6c410c66d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:04:14.015805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:14.016925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:14.018447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 630/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Jul 22 11:04:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:14.023086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a337a5-4f69-42fd-b6ef-51958cb21c51 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:14.070354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:14.127942 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a337a5-4f69-42fd-b6ef-51958cb21c51 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:14.161496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd062c6-8b11-4320-8f1a-10fce2fda143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:14.162313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd062c6-8b11-4320-8f1a-10fce2fda143 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:14.162491 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd062c6-8b11-4320-8f1a-10fce2fda143 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:14.162943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd062c6-8b11-4320-8f1a-10fce2fda143 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:14.163676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd062c6-8b11-4320-8f1a-10fce2fda143 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:14.163887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd062c6-8b11-4320-8f1a-10fce2fda143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:14.165237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd062c6-8b11-4320-8f1a-10fce2fda143 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:14.176013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 606/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Jul 22 11:04:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:14.181594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e4d6-5e41-4ee3-9564-2ab6c410c66d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:14.276613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:14.294738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e4d6-5e41-4ee3-9564-2ab6c410c66d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:14.332225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd23b3e-a34a-4d7d-b4aa-00d9a58ed606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:14.333293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd23b3e-a34a-4d7d-b4aa-00d9a58ed606 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:14.333578 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd23b3e-a34a-4d7d-b4aa-00d9a58ed606 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:14.333856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd23b3e-a34a-4d7d-b4aa-00d9a58ed606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:14.345160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd23b3e-a34a-4d7d-b4aa-00d9a58ed606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:14.347184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd23b3e-a34a-4d7d-b4aa-00d9a58ed606 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9cd23b3e-a34a-4d7d-b4aa-00d9a58ed606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:14.367054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:14.368016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:14.368945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:14.368945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:14.368945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:14.368945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:14.368945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 631/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:14.374393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd062c6-8b11-4320-8f1a-10fce2fda143 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:14.478778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd062c6-8b11-4320-8f1a-10fce2fda143 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:14.489074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:14.519539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cb403-297d-488b-adc1-bb1f9aac5756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:14.520567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cb403-297d-488b-adc1-bb1f9aac5756 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:14.520993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cb403-297d-488b-adc1-bb1f9aac5756 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:14.521411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cb403-297d-488b-adc1-bb1f9aac5756 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:14.522575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cb403-297d-488b-adc1-bb1f9aac5756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:14.523049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cb403-297d-488b-adc1-bb1f9aac5756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:14.525224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cb403-297d-488b-adc1-bb1f9aac5756 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:14.547659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 607/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:04:14.549671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd23b3e-a34a-4d7d-b4aa-00d9a58ed606 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:14.661251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd23b3e-a34a-4d7d-b4aa-00d9a58ed606 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:14.699163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73020fc0-b1ac-4bf9-993f-dd87a36c8e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:14.700060 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020fc0-b1ac-4bf9-993f-dd87a36c8e73 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:14.700385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020fc0-b1ac-4bf9-993f-dd87a36c8e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:14.700629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020fc0-b1ac-4bf9-993f-dd87a36c8e73 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:14.703526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:14.710348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73020fc0-b1ac-4bf9-993f-dd87a36c8e73 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:14.711967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73020fc0-b1ac-4bf9-993f-dd87a36c8e73 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-73020fc0-b1ac-4bf9-993f-dd87a36c8e73 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:04:14.726931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:14.728414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:14.729765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 632/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:14.734263 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cb403-297d-488b-adc1-bb1f9aac5756 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:14.850770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cb403-297d-488b-adc1-bb1f9aac5756 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:14.894043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc4890-95a7-4402-bd1f-80875b79212d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:14.894873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc4890-95a7-4402-bd1f-80875b79212d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:14.895234 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc4890-95a7-4402-bd1f-80875b79212d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:14.895847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc4890-95a7-4402-bd1f-80875b79212d None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:14.897241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc4890-95a7-4402-bd1f-80875b79212d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:14.897561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc4890-95a7-4402-bd1f-80875b79212d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:14.899030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc4890-95a7-4402-bd1f-80875b79212d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:14.921155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 608/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 22 11:04:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:14.927176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73020fc0-b1ac-4bf9-993f-dd87a36c8e73 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:15.036134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73020fc0-b1ac-4bf9-993f-dd87a36c8e73 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:15.071064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d8f7d-7878-4a1f-809d-742931a5de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:15.071782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d8f7d-7878-4a1f-809d-742931a5de6a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:15.072031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d8f7d-7878-4a1f-809d-742931a5de6a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:15.072334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d8f7d-7878-4a1f-809d-742931a5de6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:15.137081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:15.146861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d8f7d-7878-4a1f-809d-742931a5de6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:15.148364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d8f7d-7878-4a1f-809d-742931a5de6a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:15.257167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 633/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:04:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 22 11:04:15.263087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc4890-95a7-4402-bd1f-80875b79212d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:15.291884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:15.369700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc4890-95a7-4402-bd1f-80875b79212d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:15.403563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf097415-8be4-4f34-8eda-c8457119dd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:15.404369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf097415-8be4-4f34-8eda-c8457119dd1c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:15.404653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf097415-8be4-4f34-8eda-c8457119dd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:15.404932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf097415-8be4-4f34-8eda-c8457119dd1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:15.415694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf097415-8be4-4f34-8eda-c8457119dd1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:15.417147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf097415-8be4-4f34-8eda-c8457119dd1c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-bf097415-8be4-4f34-8eda-c8457119dd1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:15.435213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:15.442278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:15.443703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:15.443703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:15.443703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:15.443703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:15.443703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 609/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:15.444053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d8f7d-7878-4a1f-809d-742931a5de6a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:15.542164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:15.548057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d8f7d-7878-4a1f-809d-742931a5de6a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:15.582655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e8b2d-582b-42fc-9fbe-5b68390345c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:15.583382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e8b2d-582b-42fc-9fbe-5b68390345c1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:15.583668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e8b2d-582b-42fc-9fbe-5b68390345c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:15.583942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e8b2d-582b-42fc-9fbe-5b68390345c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:15.584705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e8b2d-582b-42fc-9fbe-5b68390345c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:15.585000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e8b2d-582b-42fc-9fbe-5b68390345c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:15.586321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e8b2d-582b-42fc-9fbe-5b68390345c1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:15.599155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 634/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:15.604824 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf097415-8be4-4f34-8eda-c8457119dd1c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:15.686258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:15.710032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf097415-8be4-4f34-8eda-c8457119dd1c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:15.746599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130e7dc-d7cd-48b7-8aba-4e868c6bdd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:15.747483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130e7dc-d7cd-48b7-8aba-4e868c6bdd4e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:15.747921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130e7dc-d7cd-48b7-8aba-4e868c6bdd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:15.748479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130e7dc-d7cd-48b7-8aba-4e868c6bdd4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:15.756964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130e7dc-d7cd-48b7-8aba-4e868c6bdd4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:15.758425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130e7dc-d7cd-48b7-8aba-4e868c6bdd4e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1130e7dc-d7cd-48b7-8aba-4e868c6bdd4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:04:15.775739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:15.776688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:15.777588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:04:15.777588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:04:15.777588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:04:15.777588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:04:15.777588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:04:15.777588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:04:15.777588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:15.777588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:15.777588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 610/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:04:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 22 11:04:15.783865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e8b2d-582b-42fc-9fbe-5b68390345c1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:15.888598 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e8b2d-582b-42fc-9fbe-5b68390345c1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:15.922395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984463a0-f217-4817-b39d-d039b0f0857c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:15.923066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984463a0-f217-4817-b39d-d039b0f0857c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:15.923376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984463a0-f217-4817-b39d-d039b0f0857c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:15.923656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984463a0-f217-4817-b39d-d039b0f0857c None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:15.924438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984463a0-f217-4817-b39d-d039b0f0857c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:15.924753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984463a0-f217-4817-b39d-d039b0f0857c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:15.926194 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984463a0-f217-4817-b39d-d039b0f0857c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:15.938376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 635/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:04:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:15.944937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130e7dc-d7cd-48b7-8aba-4e868c6bdd4e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:16.058526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130e7dc-d7cd-48b7-8aba-4e868c6bdd4e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:16.102607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598546f-5608-4c9b-a944-05a00e404467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:16.103269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598546f-5608-4c9b-a944-05a00e404467 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:16.103611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598546f-5608-4c9b-a944-05a00e404467 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:16.103863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598546f-5608-4c9b-a944-05a00e404467 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:16.104700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598546f-5608-4c9b-a944-05a00e404467 None admin] RBAC: Adding query filter params (role.id=a9580499eef24770867aeac1602aa002, scope.project.id=89224d740c7148938c2d803804083a57, user.id=88ecd4c645174d28930a44b93cfee9b6) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:16.105010 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598546f-5608-4c9b-a944-05a00e404467 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:16.106447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598546f-5608-4c9b-a944-05a00e404467 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:16.112738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 611/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:04:15 2022] GET /identity/v3/role_assignments?user.id=88ecd4c645174d28930a44b93cfee9b6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=a9580499eef24770867aeac1602aa002 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:18.348570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44e457-38c1-4441-9de5-6b552493a335 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:18.349206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44e457-38c1-4441-9de5-6b552493a335 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:18.349206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44e457-38c1-4441-9de5-6b552493a335 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:18.350503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 636/1247] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:18.356149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71757c13-1838-48bf-9acb-72d96d117f96 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:18.356686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71757c13-1838-48bf-9acb-72d96d117f96 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:18.356813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71757c13-1838-48bf-9acb-72d96d117f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:18.363147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:18.376194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-71757c13-1838-48bf-9acb-72d96d117f96 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:18.377600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-71757c13-1838-48bf-9acb-72d96d117f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:18.456422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 612/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:18.479819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44e457-38c1-4441-9de5-6b552493a335 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:18.498875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:18.591769 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44e457-38c1-4441-9de5-6b552493a335 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:18.631453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed805662-6f87-41ab-9a3b-68112a1400c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:18.632183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed805662-6f87-41ab-9a3b-68112a1400c5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:18.632472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed805662-6f87-41ab-9a3b-68112a1400c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:18.634054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed805662-6f87-41ab-9a3b-68112a1400c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:18.644062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed805662-6f87-41ab-9a3b-68112a1400c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ed805662-6f87-41ab-9a3b-68112a1400c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:04:18.652908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:18.653872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:18.654751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 637/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:04:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:18.660933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71757c13-1838-48bf-9acb-72d96d117f96 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:18.765743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71757c13-1838-48bf-9acb-72d96d117f96 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:18.799444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:18.804852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6937b-abe5-407c-b159-a16e5672cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:18.805558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6937b-abe5-407c-b159-a16e5672cb9d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:18.805849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6937b-abe5-407c-b159-a16e5672cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:18.806143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6937b-abe5-407c-b159-a16e5672cb9d None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:18.806897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6937b-abe5-407c-b159-a16e5672cb9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:18.807199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6937b-abe5-407c-b159-a16e5672cb9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:18.808615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6937b-abe5-407c-b159-a16e5672cb9d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:18.818327 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 613/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:04:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:18.824148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed805662-6f87-41ab-9a3b-68112a1400c5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:18.910943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:18.936509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed805662-6f87-41ab-9a3b-68112a1400c5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:18.970734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf343-e84c-4546-9f49-3df61a916235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:18.971610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf343-e84c-4546-9f49-3df61a916235 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:18.971852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf343-e84c-4546-9f49-3df61a916235 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:18.972126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf343-e84c-4546-9f49-3df61a916235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:18.983124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf343-e84c-4546-9f49-3df61a916235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:18.984683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf343-e84c-4546-9f49-3df61a916235 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9bbdf343-e84c-4546-9f49-3df61a916235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:19.002726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:19.003926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:19.005145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:19.005145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:19.005145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:19.005145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:19.005145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 638/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:19.010073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6937b-abe5-407c-b159-a16e5672cb9d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:19.129867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6937b-abe5-407c-b159-a16e5672cb9d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:19.169452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da835-c6ba-4559-bb76-349bd3746741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:19.170119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da835-c6ba-4559-bb76-349bd3746741 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:19.170389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da835-c6ba-4559-bb76-349bd3746741 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:19.170668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da835-c6ba-4559-bb76-349bd3746741 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:19.171480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704da835-c6ba-4559-bb76-349bd3746741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:19.171851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704da835-c6ba-4559-bb76-349bd3746741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:19.173715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704da835-c6ba-4559-bb76-349bd3746741 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:19.187397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 614/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:19.194464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf343-e84c-4546-9f49-3df61a916235 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:19.304986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf343-e84c-4546-9f49-3df61a916235 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:19.331481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:19.339694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877dccf-beaa-4618-b721-9258444b6981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:19.340519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877dccf-beaa-4618-b721-9258444b6981 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:19.340801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877dccf-beaa-4618-b721-9258444b6981 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:19.341077 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877dccf-beaa-4618-b721-9258444b6981 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:19.348954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877dccf-beaa-4618-b721-9258444b6981 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:19.350383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877dccf-beaa-4618-b721-9258444b6981 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d877dccf-beaa-4618-b721-9258444b6981 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:04:19.365123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:19.366324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:19.367534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 639/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:19.372296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da835-c6ba-4559-bb76-349bd3746741 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:19.430564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:19.478618 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da835-c6ba-4559-bb76-349bd3746741 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:19.519365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd8661-1ac1-437f-aad7-b2a657af16f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:19.520100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd8661-1ac1-437f-aad7-b2a657af16f4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:19.520382 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd8661-1ac1-437f-aad7-b2a657af16f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:19.520780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd8661-1ac1-437f-aad7-b2a657af16f4 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:19.521679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd8661-1ac1-437f-aad7-b2a657af16f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:19.521996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd8661-1ac1-437f-aad7-b2a657af16f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:19.523632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd8661-1ac1-437f-aad7-b2a657af16f4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:19.544263 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 615/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 22 11:04:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:04:19.550372 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877dccf-beaa-4618-b721-9258444b6981 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:19.661517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877dccf-beaa-4618-b721-9258444b6981 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:19.696400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7936015-acbb-42c8-b867-70cec86bcc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:19.696996 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7936015-acbb-42c8-b867-70cec86bcc45 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:19.697286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7936015-acbb-42c8-b867-70cec86bcc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:19.697567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7936015-acbb-42c8-b867-70cec86bcc45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:19.760793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:19.774186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7936015-acbb-42c8-b867-70cec86bcc45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:19.775683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7936015-acbb-42c8-b867-70cec86bcc45 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:19.882280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 640/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:04:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:04:19.889352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd8661-1ac1-437f-aad7-b2a657af16f4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:20.002435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd8661-1ac1-437f-aad7-b2a657af16f4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:20.019377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:20.044849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190a3b0-55b2-491d-8dc4-f893277ca216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:20.045616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190a3b0-55b2-491d-8dc4-f893277ca216 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:20.045906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190a3b0-55b2-491d-8dc4-f893277ca216 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:20.046204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190a3b0-55b2-491d-8dc4-f893277ca216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:20.058392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190a3b0-55b2-491d-8dc4-f893277ca216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:20.060064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190a3b0-55b2-491d-8dc4-f893277ca216 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b190a3b0-55b2-491d-8dc4-f893277ca216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:20.078197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:20.079224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:20.080677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:20.080677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:20.080677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:20.080677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:20.082041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 616/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:20.089180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7936015-acbb-42c8-b867-70cec86bcc45 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:20.190663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:20.207546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7936015-acbb-42c8-b867-70cec86bcc45 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:20.247609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c50d73-75c5-49bf-9527-957507304194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:20.248289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c50d73-75c5-49bf-9527-957507304194 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:20.248576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c50d73-75c5-49bf-9527-957507304194 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:20.248853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c50d73-75c5-49bf-9527-957507304194 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:20.249578 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c50d73-75c5-49bf-9527-957507304194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:20.249886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c50d73-75c5-49bf-9527-957507304194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:20.251351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c50d73-75c5-49bf-9527-957507304194 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:20.269589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 641/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:20.270737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190a3b0-55b2-491d-8dc4-f893277ca216 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:20.375712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190a3b0-55b2-491d-8dc4-f893277ca216 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:20.414250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753a345-230a-44d9-8308-8b5b92d4d79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:20.415496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753a345-230a-44d9-8308-8b5b92d4d79d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:20.415924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753a345-230a-44d9-8308-8b5b92d4d79d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:20.416340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753a345-230a-44d9-8308-8b5b92d4d79d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:20.427156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753a345-230a-44d9-8308-8b5b92d4d79d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:20.429468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753a345-230a-44d9-8308-8b5b92d4d79d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4753a345-230a-44d9-8308-8b5b92d4d79d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:04:20.449473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:20.450460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:20.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:04:20.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:04:20.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:04:20.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:04:20.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:04:20.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:04:20.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:20.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:20.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 617/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:04:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:20.459370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c50d73-75c5-49bf-9527-957507304194 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:20.562085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c50d73-75c5-49bf-9527-957507304194 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:20.599623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883f60b-0443-46f0-bfa4-e81233b61b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:20.600394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883f60b-0443-46f0-bfa4-e81233b61b40 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:20.600678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883f60b-0443-46f0-bfa4-e81233b61b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:20.600958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883f60b-0443-46f0-bfa4-e81233b61b40 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:20.601754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883f60b-0443-46f0-bfa4-e81233b61b40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:20.602069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883f60b-0443-46f0-bfa4-e81233b61b40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:20.603552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883f60b-0443-46f0-bfa4-e81233b61b40 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:20.616457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 642/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:04:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:04:20.622775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753a345-230a-44d9-8308-8b5b92d4d79d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:20.649702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:20.725393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753a345-230a-44d9-8308-8b5b92d4d79d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:20.759947 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44334111-2040-459a-a1ab-b103ad60a07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:20.760612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44334111-2040-459a-a1ab-b103ad60a07b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:20.760900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44334111-2040-459a-a1ab-b103ad60a07b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:20.761174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44334111-2040-459a-a1ab-b103ad60a07b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:20.762011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44334111-2040-459a-a1ab-b103ad60a07b None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=89224d740c7148938c2d803804083a57, user.id=88ecd4c645174d28930a44b93cfee9b6) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:20.762330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44334111-2040-459a-a1ab-b103ad60a07b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:20.763803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44334111-2040-459a-a1ab-b103ad60a07b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:20.770005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 618/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:04:20 2022] GET /identity/v3/role_assignments?user.id=88ecd4c645174d28930a44b93cfee9b6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:22.781139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b191b0-52b3-4a21-ab92-5a3ae735c13e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:22.781729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b191b0-52b3-4a21-ab92-5a3ae735c13e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:22.781729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b191b0-52b3-4a21-ab92-5a3ae735c13e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:22.784696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 643/1261] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:22.790042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d2c1-2c1d-4fcc-9e9b-a90e57fa0ff3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:22.790504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d2c1-2c1d-4fcc-9e9b-a90e57fa0ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:22.790613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d2c1-2c1d-4fcc-9e9b-a90e57fa0ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:22.808017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3689d2c1-2c1d-4fcc-9e9b-a90e57fa0ff3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:22.809199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3689d2c1-2c1d-4fcc-9e9b-a90e57fa0ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:22.873342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 619/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:22.895556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b191b0-52b3-4a21-ab92-5a3ae735c13e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:22.906221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:22.995435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b191b0-52b3-4a21-ab92-5a3ae735c13e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:23.027970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78c4aa-d5b1-4ada-b2b7-52225029e33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:23.028752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78c4aa-d5b1-4ada-b2b7-52225029e33c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:23.029042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78c4aa-d5b1-4ada-b2b7-52225029e33c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:23.029319 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78c4aa-d5b1-4ada-b2b7-52225029e33c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:23.039838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f78c4aa-d5b1-4ada-b2b7-52225029e33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4f78c4aa-d5b1-4ada-b2b7-52225029e33c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:04:23.047444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:23.048423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:23.049386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 644/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:04:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:23.054477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d2c1-2c1d-4fcc-9e9b-a90e57fa0ff3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:23.074307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:23.153834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d2c1-2c1d-4fcc-9e9b-a90e57fa0ff3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:23.189241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc16d6-b9a9-4eba-b046-3df368ab3388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:23.189939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc16d6-b9a9-4eba-b046-3df368ab3388 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:23.190222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc16d6-b9a9-4eba-b046-3df368ab3388 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:23.190497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc16d6-b9a9-4eba-b046-3df368ab3388 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:23.191246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc16d6-b9a9-4eba-b046-3df368ab3388 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:23.191588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc16d6-b9a9-4eba-b046-3df368ab3388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:23.192981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc16d6-b9a9-4eba-b046-3df368ab3388 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:23.201239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 620/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:04:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:23.209475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78c4aa-d5b1-4ada-b2b7-52225029e33c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:23.273718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:23.306445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78c4aa-d5b1-4ada-b2b7-52225029e33c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:23.338958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372da409-ecd7-4f8a-9e08-ae0ac44e128a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:23.339769 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372da409-ecd7-4f8a-9e08-ae0ac44e128a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:23.340057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372da409-ecd7-4f8a-9e08-ae0ac44e128a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:23.340340 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372da409-ecd7-4f8a-9e08-ae0ac44e128a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:23.350508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372da409-ecd7-4f8a-9e08-ae0ac44e128a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:23.351987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372da409-ecd7-4f8a-9e08-ae0ac44e128a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-372da409-ecd7-4f8a-9e08-ae0ac44e128a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:23.368932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:23.369983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:23.375808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:23.375808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:23.375808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:23.375808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:23.375808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 645/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:23.376660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc16d6-b9a9-4eba-b046-3df368ab3388 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:23.455370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:23.473553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc16d6-b9a9-4eba-b046-3df368ab3388 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:23.505648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a81733-d75e-4a29-972b-e5feed291c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:23.506296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a81733-d75e-4a29-972b-e5feed291c16 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:23.506583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a81733-d75e-4a29-972b-e5feed291c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:23.506860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a81733-d75e-4a29-972b-e5feed291c16 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:23.507609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a81733-d75e-4a29-972b-e5feed291c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:23.507918 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a81733-d75e-4a29-972b-e5feed291c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:23.509271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a81733-d75e-4a29-972b-e5feed291c16 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:23.521143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 621/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:23.526684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372da409-ecd7-4f8a-9e08-ae0ac44e128a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:23.622381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372da409-ecd7-4f8a-9e08-ae0ac44e128a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:23.641710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:23.656091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cc803-5ad6-41fb-8ed6-149198d61e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:23.656882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cc803-5ad6-41fb-8ed6-149198d61e83 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:23.657171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cc803-5ad6-41fb-8ed6-149198d61e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:23.657450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cc803-5ad6-41fb-8ed6-149198d61e83 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:23.665446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cc803-5ad6-41fb-8ed6-149198d61e83 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:23.666890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cc803-5ad6-41fb-8ed6-149198d61e83 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c69cc803-5ad6-41fb-8ed6-149198d61e83 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:04:23.681217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:23.682408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:23.683455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 646/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:23.688220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a81733-d75e-4a29-972b-e5feed291c16 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:23.784157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a81733-d75e-4a29-972b-e5feed291c16 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:23.816998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aebfc4-8b8b-4552-9192-09ae2a66d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:23.817652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aebfc4-8b8b-4552-9192-09ae2a66d8cc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:23.817936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aebfc4-8b8b-4552-9192-09ae2a66d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:23.818213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aebfc4-8b8b-4552-9192-09ae2a66d8cc None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:23.819046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aebfc4-8b8b-4552-9192-09ae2a66d8cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:23.819390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aebfc4-8b8b-4552-9192-09ae2a66d8cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:23.820822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aebfc4-8b8b-4552-9192-09ae2a66d8cc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:23.829144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:23.839344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 622/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 22 11:04:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:23.844880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cc803-5ad6-41fb-8ed6-149198d61e83 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:23.942032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cc803-5ad6-41fb-8ed6-149198d61e83 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:23.973441 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08611ca-1f28-4bd8-b603-772e543c5ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:23.974210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08611ca-1f28-4bd8-b603-772e543c5ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:23.974493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08611ca-1f28-4bd8-b603-772e543c5ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:23.974766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08611ca-1f28-4bd8-b603-772e543c5ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:23.985078 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08611ca-1f28-4bd8-b603-772e543c5ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:23.987287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08611ca-1f28-4bd8-b603-772e543c5ff0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b08611ca-1f28-4bd8-b603-772e543c5ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:24.004303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:24.005366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:24.006471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:24.006471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:24.006471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:24.006471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:24.006471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 647/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:24.010925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aebfc4-8b8b-4552-9192-09ae2a66d8cc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:24.106964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aebfc4-8b8b-4552-9192-09ae2a66d8cc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:24.140373 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c99fd-a0cb-4bfb-8f0b-1dfb5235c72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:24.141028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c99fd-a0cb-4bfb-8f0b-1dfb5235c72f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:24.141315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c99fd-a0cb-4bfb-8f0b-1dfb5235c72f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:24.141595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c99fd-a0cb-4bfb-8f0b-1dfb5235c72f None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:24.142344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c99fd-a0cb-4bfb-8f0b-1dfb5235c72f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:24.142654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c99fd-a0cb-4bfb-8f0b-1dfb5235c72f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:24.144146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c99fd-a0cb-4bfb-8f0b-1dfb5235c72f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:24.155788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 623/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:24.162112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08611ca-1f28-4bd8-b603-772e543c5ff0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:24.197810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:24.264205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08611ca-1f28-4bd8-b603-772e543c5ff0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:24.296292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e356fd-8f9f-4932-b904-cb135b3721db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:24.297072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e356fd-8f9f-4932-b904-cb135b3721db None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:24.297354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e356fd-8f9f-4932-b904-cb135b3721db None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:24.297629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e356fd-8f9f-4932-b904-cb135b3721db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:24.304685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e356fd-8f9f-4932-b904-cb135b3721db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:24.306113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e356fd-8f9f-4932-b904-cb135b3721db None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-41e356fd-8f9f-4932-b904-cb135b3721db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:04:24.319858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:24.320903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:24.322015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:04:24.322015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:04:24.322015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:04:24.322015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:04:24.322015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:04:24.322015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:04:24.322015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:24.322015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:24.322015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 648/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:04:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:24.326805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c99fd-a0cb-4bfb-8f0b-1dfb5235c72f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:24.383225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:24.422754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c99fd-a0cb-4bfb-8f0b-1dfb5235c72f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:24.455169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cf191-5674-4c4f-9152-326574fb5614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:24.455975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cf191-5674-4c4f-9152-326574fb5614 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:24.456257 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cf191-5674-4c4f-9152-326574fb5614 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:24.456526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cf191-5674-4c4f-9152-326574fb5614 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:24.457303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cf191-5674-4c4f-9152-326574fb5614 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:24.457609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cf191-5674-4c4f-9152-326574fb5614 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:24.459023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cf191-5674-4c4f-9152-326574fb5614 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:24.471143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 624/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:04:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:24.476967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e356fd-8f9f-4932-b904-cb135b3721db None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:24.563463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:24.572220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e356fd-8f9f-4932-b904-cb135b3721db None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:24.605783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb7748-8d05-4b55-8468-9ef0892290a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:24.606676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb7748-8d05-4b55-8468-9ef0892290a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:24.606962 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb7748-8d05-4b55-8468-9ef0892290a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:24.607242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb7748-8d05-4b55-8468-9ef0892290a0 None admin] PATH_INFO: `/v3/projects/89224d740c7148938c2d803804083a57/users/88ecd4c645174d28930a44b93cfee9b6/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:24.609191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb7748-8d05-4b55-8468-9ef0892290a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=89224d740c7148938c2d803804083a57, user_id=88ecd4c645174d28930a44b93cfee9b6, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:24.610650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb7748-8d05-4b55-8468-9ef0892290a0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:24.622877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 649/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Jul 22 11:04:24 2022] PUT /identity/v3/projects/89224d740c7148938c2d803804083a57/users/88ecd4c645174d28930a44b93cfee9b6/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:04:26.631202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e4fe7-b4d1-446e-93ae-65b7751f6c6f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:26.631717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e4fe7-b4d1-446e-93ae-65b7751f6c6f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:26.631816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e4fe7-b4d1-446e-93ae-65b7751f6c6f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:26.633172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 625/1274] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:26.638318 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c05bd9-4599-4159-8f14-173539f512c2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:26.638752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c05bd9-4599-4159-8f14-173539f512c2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:26.638856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c05bd9-4599-4159-8f14-173539f512c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:26.655808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-55c05bd9-4599-4159-8f14-173539f512c2 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:26.656978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-55c05bd9-4599-4159-8f14-173539f512c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:26.722857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 650/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:26.746438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e4fe7-b4d1-446e-93ae-65b7751f6c6f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:26.845114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e4fe7-b4d1-446e-93ae-65b7751f6c6f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:26.862390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:26.879654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab320e3-c39f-48ad-bf4e-301c4d755e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:26.880450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab320e3-c39f-48ad-bf4e-301c4d755e0e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:26.880736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab320e3-c39f-48ad-bf4e-301c4d755e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:26.881015 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab320e3-c39f-48ad-bf4e-301c4d755e0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:26.890945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab320e3-c39f-48ad-bf4e-301c4d755e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8ab320e3-c39f-48ad-bf4e-301c4d755e0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 22 11:04:26.898549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:26.899545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:26.900426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 626/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Jul 22 11:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:26.905749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c05bd9-4599-4159-8f14-173539f512c2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:26.982887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:27.002978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c05bd9-4599-4159-8f14-173539f512c2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:27.036148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3534-cf9b-4d33-956e-a2d3b02545a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:27.036806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3534-cf9b-4d33-956e-a2d3b02545a6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:27.037091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3534-cf9b-4d33-956e-a2d3b02545a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:27.037369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3534-cf9b-4d33-956e-a2d3b02545a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:27.038106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d3534-cf9b-4d33-956e-a2d3b02545a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:27.038412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d3534-cf9b-4d33-956e-a2d3b02545a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:27.039813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d3534-cf9b-4d33-956e-a2d3b02545a6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:27.047696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 651/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Jul 22 11:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:27.053995 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab320e3-c39f-48ad-bf4e-301c4d755e0e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:27.152550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab320e3-c39f-48ad-bf4e-301c4d755e0e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:27.185753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f0d75-7973-49db-a36c-1c41c40280e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:27.186534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f0d75-7973-49db-a36c-1c41c40280e6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:27.186815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f0d75-7973-49db-a36c-1c41c40280e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:27.187089 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f0d75-7973-49db-a36c-1c41c40280e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:27.197332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f0d75-7973-49db-a36c-1c41c40280e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:27.198760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f0d75-7973-49db-a36c-1c41c40280e6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-772f0d75-7973-49db-a36c-1c41c40280e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:27.215117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:27.216153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:27.217122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:27.217122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:27.217122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:27.217122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:27.217122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 627/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:27.222778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3534-cf9b-4d33-956e-a2d3b02545a6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:27.319311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3534-cf9b-4d33-956e-a2d3b02545a6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:27.351385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f1b62-5192-4285-bd8d-0d8473fb3b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:27.352036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f1b62-5192-4285-bd8d-0d8473fb3b15 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:27.352323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f1b62-5192-4285-bd8d-0d8473fb3b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:27.352604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f1b62-5192-4285-bd8d-0d8473fb3b15 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:27.353321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f1b62-5192-4285-bd8d-0d8473fb3b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:27.353629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f1b62-5192-4285-bd8d-0d8473fb3b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:27.354989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f1b62-5192-4285-bd8d-0d8473fb3b15 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:27.367158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 652/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:27.372933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f0d75-7973-49db-a36c-1c41c40280e6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:27.402849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:27.472999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f0d75-7973-49db-a36c-1c41c40280e6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:27.506467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4e35f-9ade-4bf2-8abc-76f88c211783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:27.507290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4e35f-9ade-4bf2-8abc-76f88c211783 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:27.507587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4e35f-9ade-4bf2-8abc-76f88c211783 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:27.507864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4e35f-9ade-4bf2-8abc-76f88c211783 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:27.515244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4e35f-9ade-4bf2-8abc-76f88c211783 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:27.517250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4e35f-9ade-4bf2-8abc-76f88c211783 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-86c4e35f-9ade-4bf2-8abc-76f88c211783 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 22 11:04:27.531269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:27.532927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:27.534087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 628/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:27.539073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f1b62-5192-4285-bd8d-0d8473fb3b15 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:27.542773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:27.634493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f1b62-5192-4285-bd8d-0d8473fb3b15 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:27.667376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d59501-37f2-4a94-98d3-253f06055461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:27.667973 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d59501-37f2-4a94-98d3-253f06055461 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:27.668256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d59501-37f2-4a94-98d3-253f06055461 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:27.668533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d59501-37f2-4a94-98d3-253f06055461 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:27.669358 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d59501-37f2-4a94-98d3-253f06055461 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:27.669666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d59501-37f2-4a94-98d3-253f06055461 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:27.671075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d59501-37f2-4a94-98d3-253f06055461 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:27.687320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 653/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Jul 22 11:04:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:27.694851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4e35f-9ade-4bf2-8abc-76f88c211783 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:27.782952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:27.792661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4e35f-9ade-4bf2-8abc-76f88c211783 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:27.824562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae9ff1-cd12-4094-ac27-36fffac9ad5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:27.825195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae9ff1-cd12-4094-ac27-36fffac9ad5a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:27.825481 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae9ff1-cd12-4094-ac27-36fffac9ad5a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:27.825758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae9ff1-cd12-4094-ac27-36fffac9ad5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:27.889860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae9ff1-cd12-4094-ac27-36fffac9ad5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:27.891315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae9ff1-cd12-4094-ac27-36fffac9ad5a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:27.975582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:27.985803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 629/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Jul 22 11:04:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 22 11:04:27.991809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d59501-37f2-4a94-98d3-253f06055461 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:28.050012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:28.088238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d59501-37f2-4a94-98d3-253f06055461 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:28.121067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317157d-be4c-48c5-8f3c-ac34dd6cb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:28.121859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317157d-be4c-48c5-8f3c-ac34dd6cb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:28.122147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317157d-be4c-48c5-8f3c-ac34dd6cb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:28.122452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317157d-be4c-48c5-8f3c-ac34dd6cb2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:28.132578 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317157d-be4c-48c5-8f3c-ac34dd6cb2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:28.134011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317157d-be4c-48c5-8f3c-ac34dd6cb2b7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4317157d-be4c-48c5-8f3c-ac34dd6cb2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 22 11:04:28.151396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:28.152453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:28.153716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 22 11:04:28.153716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 22 11:04:28.153716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 22 11:04:28.153716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:28.153716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 654/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:28.158214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae9ff1-cd12-4094-ac27-36fffac9ad5a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:28.253586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae9ff1-cd12-4094-ac27-36fffac9ad5a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:28.290207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc8f3a-f7cf-41ce-81fc-09d8c9ae6299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:28.290861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc8f3a-f7cf-41ce-81fc-09d8c9ae6299 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:28.291148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc8f3a-f7cf-41ce-81fc-09d8c9ae6299 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:28.291454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc8f3a-f7cf-41ce-81fc-09d8c9ae6299 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:28.292175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc8f3a-f7cf-41ce-81fc-09d8c9ae6299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:28.292482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc8f3a-f7cf-41ce-81fc-09d8c9ae6299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:28.293835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc8f3a-f7cf-41ce-81fc-09d8c9ae6299 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:28.305280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 630/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Jul 22 11:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:04:28.310789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317157d-be4c-48c5-8f3c-ac34dd6cb2b7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:28.405564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317157d-be4c-48c5-8f3c-ac34dd6cb2b7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:28.418839 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:28.439440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef63867-7986-471a-8d15-c034be4c19b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:28.440215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef63867-7986-471a-8d15-c034be4c19b1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:28.440498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef63867-7986-471a-8d15-c034be4c19b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:28.440776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef63867-7986-471a-8d15-c034be4c19b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:28.447204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef63867-7986-471a-8d15-c034be4c19b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:28.448684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef63867-7986-471a-8d15-c034be4c19b1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-cef63867-7986-471a-8d15-c034be4c19b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 22 11:04:28.461932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:28.463041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:28.464259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 22 11:04:28.464259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 22 11:04:28.464259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 22 11:04:28.464259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 22 11:04:28.464259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 22 11:04:28.464259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 22 11:04:28.464259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 22 11:04:28.464259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 22 11:04:28.464259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 655/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Jul 22 11:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:28.468708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc8f3a-f7cf-41ce-81fc-09d8c9ae6299 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:28.518041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:28.567694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc8f3a-f7cf-41ce-81fc-09d8c9ae6299 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:28.600633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420ab78-fe25-40cf-9e61-b0728e9a3155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:28.601296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420ab78-fe25-40cf-9e61-b0728e9a3155 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:28.601581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420ab78-fe25-40cf-9e61-b0728e9a3155 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:28.601860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420ab78-fe25-40cf-9e61-b0728e9a3155 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:28.602645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420ab78-fe25-40cf-9e61-b0728e9a3155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:28.602955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420ab78-fe25-40cf-9e61-b0728e9a3155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:28.604432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420ab78-fe25-40cf-9e61-b0728e9a3155 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:28.617350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 631/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Jul 22 11:04:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:28.623354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef63867-7986-471a-8d15-c034be4c19b1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:28.720608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef63867-7986-471a-8d15-c034be4c19b1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:28.752864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde4d39-81dc-451b-8a39-9122af1b7a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:28.753506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde4d39-81dc-451b-8a39-9122af1b7a93 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:28.753790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde4d39-81dc-451b-8a39-9122af1b7a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:28.754062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde4d39-81dc-451b-8a39-9122af1b7a93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:28.754888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde4d39-81dc-451b-8a39-9122af1b7a93 None admin] RBAC: Adding query filter params (role.id=7f92cc1bdd9240b887d0f2aaf4055734, scope.project.id=89224d740c7148938c2d803804083a57, user.id=88ecd4c645174d28930a44b93cfee9b6) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:28.755199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde4d39-81dc-451b-8a39-9122af1b7a93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:28.756642 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde4d39-81dc-451b-8a39-9122af1b7a93 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:28.762202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 656/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Jul 22 11:04:28 2022] GET /identity/v3/role_assignments?user.id=88ecd4c645174d28930a44b93cfee9b6&scope.project.id=89224d740c7148938c2d803804083a57&role.id=7f92cc1bdd9240b887d0f2aaf4055734 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:30.809381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b7435-d2e4-444e-a31a-c51c492532bb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:30.809929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b7435-d2e4-444e-a31a-c51c492532bb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:30.809929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b7435-d2e4-444e-a31a-c51c492532bb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:30.811321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 632/1288] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:30.817931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6cc6d-8ae7-487d-9437-a18875066594 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:30.818441 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6cc6d-8ae7-487d-9437-a18875066594 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:30.818942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6cc6d-8ae7-487d-9437-a18875066594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:30.837950 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b8a6cc6d-8ae7-487d-9437-a18875066594 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:30.839176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b8a6cc6d-8ae7-487d-9437-a18875066594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:30.867833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:30.905044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 657/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:30.927022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b7435-d2e4-444e-a31a-c51c492532bb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:31.024738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b7435-d2e4-444e-a31a-c51c492532bb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:31.034890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:31.057458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c57c60-00fb-446f-b8b5-8b92aa2fe4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:31.058239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c57c60-00fb-446f-b8b5-8b92aa2fe4eb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:31.058520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c57c60-00fb-446f-b8b5-8b92aa2fe4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:31.058797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c57c60-00fb-446f-b8b5-8b92aa2fe4eb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:31.059468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c57c60-00fb-446f-b8b5-8b92aa2fe4eb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:31.060835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c57c60-00fb-446f-b8b5-8b92aa2fe4eb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f1c57c60-00fb-446f-b8b5-8b92aa2fe4eb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:31.072817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:31.074062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:31.074940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 633/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:04:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:31.080466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6cc6d-8ae7-487d-9437-a18875066594 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:31.189248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6cc6d-8ae7-487d-9437-a18875066594 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:31.223931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b14b3-f5f9-4631-b73d-91afae4e877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:31.224588 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b14b3-f5f9-4631-b73d-91afae4e877a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:31.224874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b14b3-f5f9-4631-b73d-91afae4e877a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:31.225149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b14b3-f5f9-4631-b73d-91afae4e877a None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:31.225865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b14b3-f5f9-4631-b73d-91afae4e877a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:31.226170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b14b3-f5f9-4631-b73d-91afae4e877a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:31.227550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b14b3-f5f9-4631-b73d-91afae4e877a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:31.232239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 658/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Jul 22 11:04:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:31.238918 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c57c60-00fb-446f-b8b5-8b92aa2fe4eb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:31.335925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c57c60-00fb-446f-b8b5-8b92aa2fe4eb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:31.368750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f197d94-7f66-47bf-8839-0c37885fb8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:31.369406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f197d94-7f66-47bf-8839-0c37885fb8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:31.369691 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f197d94-7f66-47bf-8839-0c37885fb8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:31.369968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f197d94-7f66-47bf-8839-0c37885fb8b6 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:31.370699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f197d94-7f66-47bf-8839-0c37885fb8b6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:31.371008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f197d94-7f66-47bf-8839-0c37885fb8b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:31.372410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f197d94-7f66-47bf-8839-0c37885fb8b6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:31.377376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 634/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Jul 22 11:04:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:33.439275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392364bb-ecee-4b6b-8b32-2f22a89286f6 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:33.439948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392364bb-ecee-4b6b-8b32-2f22a89286f6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:33.439948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392364bb-ecee-4b6b-8b32-2f22a89286f6 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:33.441248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 659/1293] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:33.446491 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f8e0f-a3f7-4305-bc56-848a5eeb1199 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:33.446900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f8e0f-a3f7-4305-bc56-848a5eeb1199 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:33.447055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f8e0f-a3f7-4305-bc56-848a5eeb1199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:33.463843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-311f8e0f-a3f7-4305-bc56-848a5eeb1199 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:33.465044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-311f8e0f-a3f7-4305-bc56-848a5eeb1199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:33.494960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:33.533384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 635/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:33.555720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392364bb-ecee-4b6b-8b32-2f22a89286f6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:33.586051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:33.654772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392364bb-ecee-4b6b-8b32-2f22a89286f6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:33.686756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533827e7-346f-4270-b7d6-c330c7e9444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:33.687433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533827e7-346f-4270-b7d6-c330c7e9444f None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:33.687722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533827e7-346f-4270-b7d6-c330c7e9444f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:33.687998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533827e7-346f-4270-b7d6-c330c7e9444f None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:33.688729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533827e7-346f-4270-b7d6-c330c7e9444f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:33.690126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533827e7-346f-4270-b7d6-c330c7e9444f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:33.702803 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 660/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Jul 22 11:04:33 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:04:35.733967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458ef9-c051-4f2a-9352-97906370665f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:35.734501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458ef9-c051-4f2a-9352-97906370665f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:35.734595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458ef9-c051-4f2a-9352-97906370665f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:35.736710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 636/1296] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:35.741968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0af7fb-703d-4b52-ac52-999f08597f10 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:35.742348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0af7fb-703d-4b52-ac52-999f08597f10 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:35.742535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0af7fb-703d-4b52-ac52-999f08597f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:35.759419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cc0af7fb-703d-4b52-ac52-999f08597f10 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:35.760613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cc0af7fb-703d-4b52-ac52-999f08597f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:35.830338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 661/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:35.852837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15458ef9-c051-4f2a-9352-97906370665f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:35.952979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15458ef9-c051-4f2a-9352-97906370665f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:35.985596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fd2bd-73d3-4a9c-82c8-352b4262d274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:35.986385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fd2bd-73d3-4a9c-82c8-352b4262d274 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:35.986933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fd2bd-73d3-4a9c-82c8-352b4262d274 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:35.987208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fd2bd-73d3-4a9c-82c8-352b4262d274 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:35.987884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fd2bd-73d3-4a9c-82c8-352b4262d274 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:35.989255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fd2bd-73d3-4a9c-82c8-352b4262d274 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9b3fd2bd-73d3-4a9c-82c8-352b4262d274 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:36.001492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:36.002677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:36.003675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 637/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:04:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:36.008733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0af7fb-703d-4b52-ac52-999f08597f10 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:36.026428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:36.104529 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0af7fb-703d-4b52-ac52-999f08597f10 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:36.136937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3826ad6-3a1e-46b5-a606-4cab93fe268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:36.137595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3826ad6-3a1e-46b5-a606-4cab93fe268e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:36.137883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3826ad6-3a1e-46b5-a606-4cab93fe268e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:36.138164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3826ad6-3a1e-46b5-a606-4cab93fe268e None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:36.138862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3826ad6-3a1e-46b5-a606-4cab93fe268e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:36.139171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3826ad6-3a1e-46b5-a606-4cab93fe268e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:36.140556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3826ad6-3a1e-46b5-a606-4cab93fe268e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:36.145562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 662/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Jul 22 11:04:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:36.152133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fd2bd-73d3-4a9c-82c8-352b4262d274 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:36.191117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:36.258145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fd2bd-73d3-4a9c-82c8-352b4262d274 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:36.289864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b795d-b83e-42f9-bfba-86a5673b0ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:36.290514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b795d-b83e-42f9-bfba-86a5673b0ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:36.290797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b795d-b83e-42f9-bfba-86a5673b0ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:36.291071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b795d-b83e-42f9-bfba-86a5673b0ba5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:36.291880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b795d-b83e-42f9-bfba-86a5673b0ba5 None admin] RBAC: Adding query filter params (interface=public, service_id=f21426bac5e847c2b9a9d26b57651728, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:36.292190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b795d-b83e-42f9-bfba-86a5673b0ba5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:36.293542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b795d-b83e-42f9-bfba-86a5673b0ba5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:36.298928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 638/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Jul 22 11:04:36 2022] GET /identity/v3/endpoints?service_id=f21426bac5e847c2b9a9d26b57651728&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:36.304589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3826ad6-3a1e-46b5-a606-4cab93fe268e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:36.383400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:36.401142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3826ad6-3a1e-46b5-a606-4cab93fe268e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:36.434092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4321d-fea4-4ede-a9e6-e6fbdf7bbd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:36.434744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4321d-fea4-4ede-a9e6-e6fbdf7bbd71 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:36.435029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4321d-fea4-4ede-a9e6-e6fbdf7bbd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:36.435334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4321d-fea4-4ede-a9e6-e6fbdf7bbd71 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:36.436005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4321d-fea4-4ede-a9e6-e6fbdf7bbd71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:36.437524 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4321d-fea4-4ede-a9e6-e6fbdf7bbd71 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:36.444252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 663/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Jul 22 11:04:36 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:04:38.421759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066970f-a306-48cc-8104-15ec9a3e7913 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:38.422294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066970f-a306-48cc-8104-15ec9a3e7913 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:38.422389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066970f-a306-48cc-8104-15ec9a3e7913 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:38.423755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 639/1302] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:04:38.428868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87df1b-62e8-45be-84f0-d0d0965a592b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:38.429312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87df1b-62e8-45be-84f0-d0d0965a592b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:38.429406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87df1b-62e8-45be-84f0-d0d0965a592b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:38.446091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ee87df1b-62e8-45be-84f0-d0d0965a592b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:04:38.447274 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ee87df1b-62e8-45be-84f0-d0d0965a592b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:04:38.511515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 664/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:04:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 22 11:04:38.535432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066970f-a306-48cc-8104-15ec9a3e7913 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:38.629928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:38.637242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066970f-a306-48cc-8104-15ec9a3e7913 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:38.673362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb4bd3-1c3c-4d33-9ec6-5af1a7dd1901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:38.674091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb4bd3-1c3c-4d33-9ec6-5af1a7dd1901 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:38.674372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb4bd3-1c3c-4d33-9ec6-5af1a7dd1901 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:38.674642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb4bd3-1c3c-4d33-9ec6-5af1a7dd1901 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:38.675300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb4bd3-1c3c-4d33-9ec6-5af1a7dd1901 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:38.676668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb4bd3-1c3c-4d33-9ec6-5af1a7dd1901 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6beb4bd3-1c3c-4d33-9ec6-5af1a7dd1901 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 22 11:04:38.688854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 22 11:04:38.690000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 22 11:04:38.691184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 640/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Jul 22 11:04:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 22 11:04:38.696009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87df1b-62e8-45be-84f0-d0d0965a592b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:38.768137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:38.789895 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87df1b-62e8-45be-84f0-d0d0965a592b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:38.822645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124377-57d2-4dec-a0b4-c227e56d222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:38.823357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124377-57d2-4dec-a0b4-c227e56d222a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:38.823647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124377-57d2-4dec-a0b4-c227e56d222a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:38.823933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124377-57d2-4dec-a0b4-c227e56d222a None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:38.824629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea124377-57d2-4dec-a0b4-c227e56d222a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:04:38.824937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea124377-57d2-4dec-a0b4-c227e56d222a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:38.826299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea124377-57d2-4dec-a0b4-c227e56d222a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:38.831317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 665/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Jul 22 11:04:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:04:38.836857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb4bd3-1c3c-4d33-9ec6-5af1a7dd1901 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:04:38.935596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb4bd3-1c3c-4d33-9ec6-5af1a7dd1901 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:04:38.967826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e175b76-d82b-4b6e-82e0-1ad747f5b12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:04:38.968484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e175b76-d82b-4b6e-82e0-1ad747f5b12d None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:04:38.968775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e175b76-d82b-4b6e-82e0-1ad747f5b12d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:04:38.969054 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e175b76-d82b-4b6e-82e0-1ad747f5b12d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:04:38.969794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e175b76-d82b-4b6e-82e0-1ad747f5b12d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:04:38.971229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e175b76-d82b-4b6e-82e0-1ad747f5b12d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:04:38.989613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 641/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Jul 22 11:04:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:04:48.828343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=30 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:04:48.984668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:41.390626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381b357-2273-4b29-a8b7-8e5f6db03ce8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:41.391766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381b357-2273-4b29-a8b7-8e5f6db03ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:41.392284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381b357-2273-4b29-a8b7-8e5f6db03ce8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:41.398015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 666/1307] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:05:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:05:41.400007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f99289-c5b6-4276-8362-1884d336d536 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:41.400899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f99289-c5b6-4276-8362-1884d336d536 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:41.401449 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f99289-c5b6-4276-8362-1884d336d536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:41.484779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d2f99289-c5b6-4276-8362-1884d336d536 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:05:41.485914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d2f99289-c5b6-4276-8362-1884d336d536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:41.603281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 642/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:05:41.647896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3097b68-fb33-4123-b7e7-3a337771310d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:41.648517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3097b68-fb33-4123-b7e7-3a337771310d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:41.649014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3097b68-fb33-4123-b7e7-3a337771310d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:41.650649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 667/1309] 10.222.0.20 () {54 vars in 965 bytes} [Fri Jul 22 11:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:05:41.667172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9cd82-0b35-4653-98b5-812272050cd1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:41.667523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9cd82-0b35-4653-98b5-812272050cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:41.667786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9cd82-0b35-4653-98b5-812272050cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:41.800898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bef9cd82-0b35-4653-98b5-812272050cd1 None None] MFA Rules not processed for user `0a0ce1804b514b42845ab33c7bfee309`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:05:41.802105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bef9cd82-0b35-4653-98b5-812272050cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:41.840268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:41.907115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 643/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Fri Jul 22 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:05:41.916914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3097b68-fb33-4123-b7e7-3a337771310d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:05:41.921411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b3097b68-fb33-4123-b7e7-3a337771310d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:42.135461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3097b68-fb33-4123-b7e7-3a337771310d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:05:42.239497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41ce2a-eca1-4592-9d9b-3ab09e2d5fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:05:42.240072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41ce2a-eca1-4592-9d9b-3ab09e2d5fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:42.240362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41ce2a-eca1-4592-9d9b-3ab09e2d5fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:42.240638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41ce2a-eca1-4592-9d9b-3ab09e2d5fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:42.242312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2b41ce2a-eca1-4592-9d9b-3ab09e2d5fa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:42.303932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:42.364365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41ce2a-eca1-4592-9d9b-3ab09e2d5fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:05:42.365815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41ce2a-eca1-4592-9d9b-3ab09e2d5fa4 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:05:42.385894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 668/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:05:44.723729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe36a1e-8421-48cf-b911-75aeeff9a0da None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:44.724272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe36a1e-8421-48cf-b911-75aeeff9a0da None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:44.724272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe36a1e-8421-48cf-b911-75aeeff9a0da None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:44.725655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 644/1312] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:05:44.730701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53e265-6b3b-4323-8f86-51a9fab27c60 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:44.731086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53e265-6b3b-4323-8f86-51a9fab27c60 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:44.731432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53e265-6b3b-4323-8f86-51a9fab27c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:44.812560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4c53e265-6b3b-4323-8f86-51a9fab27c60 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:05:44.814054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4c53e265-6b3b-4323-8f86-51a9fab27c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:44.913196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 669/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:05:44.951412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe36a1e-8421-48cf-b911-75aeeff9a0da None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:05:44.952965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8fe36a1e-8421-48cf-b911-75aeeff9a0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:45.067796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe36a1e-8421-48cf-b911-75aeeff9a0da None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:05:45.145232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:45.147798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abd99a-fe6a-433b-a1ef-479ebb5cfdb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:05:45.148559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abd99a-fe6a-433b-a1ef-479ebb5cfdb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:45.148930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abd99a-fe6a-433b-a1ef-479ebb5cfdb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:45.149260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abd99a-fe6a-433b-a1ef-479ebb5cfdb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:45.151479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-69abd99a-fe6a-433b-a1ef-479ebb5cfdb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:45.258119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abd99a-fe6a-433b-a1ef-479ebb5cfdb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:05:45.259514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abd99a-fe6a-433b-a1ef-479ebb5cfdb7 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:05:45.279281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 645/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:05:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:05:47.597396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e76533-f675-4d0e-98c9-8aa8a76233a9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:47.597785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e76533-f675-4d0e-98c9-8aa8a76233a9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:47.597963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e76533-f675-4d0e-98c9-8aa8a76233a9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:47.599426 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 670/1315] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:05:47.604937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaa95f-5722-4045-bd90-2f16e4f27833 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:47.605397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaa95f-5722-4045-bd90-2f16e4f27833 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:47.605490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaa95f-5722-4045-bd90-2f16e4f27833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:47.675600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fbbaa95f-5722-4045-bd90-2f16e4f27833 None None] MFA Rules not processed for user `6e889b4f5d384fef831673b867e27bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:05:47.676738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fbbaa95f-5722-4045-bd90-2f16e4f27833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:47.726597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:47.784983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 646/1316] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:05:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:05:47.825914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e76533-f675-4d0e-98c9-8aa8a76233a9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:05:47.827515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-69e76533-f675-4d0e-98c9-8aa8a76233a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:47.837188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:47.932624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e76533-f675-4d0e-98c9-8aa8a76233a9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:05:48.008699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54defbe-7b1d-4db6-b56f-4ea8d7abe271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:05:48.009380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54defbe-7b1d-4db6-b56f-4ea8d7abe271 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:48.009671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54defbe-7b1d-4db6-b56f-4ea8d7abe271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:48.009951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54defbe-7b1d-4db6-b56f-4ea8d7abe271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:48.011637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a54defbe-7b1d-4db6-b56f-4ea8d7abe271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:48.074095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:48.120355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54defbe-7b1d-4db6-b56f-4ea8d7abe271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:05:48.121794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54defbe-7b1d-4db6-b56f-4ea8d7abe271 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:05:48.139745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 671/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:05:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:05:50.721890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a7672-a9e2-42aa-9e3e-695ac077910e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:50.722446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a7672-a9e2-42aa-9e3e-695ac077910e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:50.722446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a7672-a9e2-42aa-9e3e-695ac077910e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:50.723896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 647/1318] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:05:50.729120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe3dc1-9ec6-43cb-866e-1aa4c10b0c3f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:50.729538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe3dc1-9ec6-43cb-866e-1aa4c10b0c3f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:50.729678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe3dc1-9ec6-43cb-866e-1aa4c10b0c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:50.792545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1cfe3dc1-9ec6-43cb-866e-1aa4c10b0c3f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:05:50.793754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1cfe3dc1-9ec6-43cb-866e-1aa4c10b0c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:50.888356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 672/1319] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:05:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:05:50.946066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95abb2e-ebcb-4077-9465-660968c924e5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:50.946365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95abb2e-ebcb-4077-9465-660968c924e5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:50.946734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95abb2e-ebcb-4077-9465-660968c924e5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:50.949299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 648/1320] 10.222.0.20 () {54 vars in 965 bytes} [Fri Jul 22 11:05:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:05:50.955272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afde41-f191-4171-8628-f8d2603f60dc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:50.955573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afde41-f191-4171-8628-f8d2603f60dc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:50.955854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afde41-f191-4171-8628-f8d2603f60dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:50.976553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:51.029887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-31afde41-f191-4171-8628-f8d2603f60dc None None] MFA Rules not processed for user `0a0ce1804b514b42845ab33c7bfee309`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:05:51.031068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-31afde41-f191-4171-8628-f8d2603f60dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:51.104707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 673/1321] 10.222.0.20 () {58 vars in 1062 bytes} [Fri Jul 22 11:05:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:05:51.114349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95abb2e-ebcb-4077-9465-660968c924e5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:05:51.115906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d95abb2e-ebcb-4077-9465-660968c924e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:51.219932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95abb2e-ebcb-4077-9465-660968c924e5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:05:51.290165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049171e-09e1-46fe-af8e-6bd702000efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:05:51.290807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049171e-09e1-46fe-af8e-6bd702000efe service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:51.291091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049171e-09e1-46fe-af8e-6bd702000efe service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:51.291401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049171e-09e1-46fe-af8e-6bd702000efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:51.293037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d049171e-09e1-46fe-af8e-6bd702000efe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:51.301895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:51.400051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049171e-09e1-46fe-af8e-6bd702000efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:05:51.401504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049171e-09e1-46fe-af8e-6bd702000efe service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:05:51.420113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 649/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:05:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:05:54.617171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70437f9-5338-4de1-b078-ac04401955d0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:54.617757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70437f9-5338-4de1-b078-ac04401955d0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:54.617757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70437f9-5338-4de1-b078-ac04401955d0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:54.619100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 674/1323] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:05:54.624586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195a2ba-cfce-4213-9a4f-ee57d3c7fd05 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:54.625026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195a2ba-cfce-4213-9a4f-ee57d3c7fd05 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:54.625155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195a2ba-cfce-4213-9a4f-ee57d3c7fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:54.696211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2195a2ba-cfce-4213-9a4f-ee57d3c7fd05 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:05:54.697453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2195a2ba-cfce-4213-9a4f-ee57d3c7fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:54.765842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:54.812742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 650/1324] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:05:54.850753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70437f9-5338-4de1-b078-ac04401955d0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:05:54.852362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a70437f9-5338-4de1-b078-ac04401955d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:54.961038 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70437f9-5338-4de1-b078-ac04401955d0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:05:54.973390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:55.039753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a1105-1d82-4725-becd-cf0391522f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:05:55.040406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a1105-1d82-4725-becd-cf0391522f5e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:55.040688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a1105-1d82-4725-becd-cf0391522f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:55.040961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a1105-1d82-4725-becd-cf0391522f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:55.042605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3f9a1105-1d82-4725-becd-cf0391522f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:55.153809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a1105-1d82-4725-becd-cf0391522f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:05:55.155141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a1105-1d82-4725-becd-cf0391522f5e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:05:55.178334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 675/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:05:57.255038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a369d-c5bf-41c9-b9d5-8e678168f689 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:57.255678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a369d-c5bf-41c9-b9d5-8e678168f689 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:57.255678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a369d-c5bf-41c9-b9d5-8e678168f689 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:57.256897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 651/1326] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:05:57.262201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bd0b-d24f-4a45-a31e-d5b4dc7e3da8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:57.262616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bd0b-d24f-4a45-a31e-d5b4dc7e3da8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:57.263446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bd0b-d24f-4a45-a31e-d5b4dc7e3da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:57.308400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:57.327038 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-db67bd0b-d24f-4a45-a31e-d5b4dc7e3da8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:05:57.328276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-db67bd0b-d24f-4a45-a31e-d5b4dc7e3da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:57.425330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 676/1327] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:05:57.461828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a369d-c5bf-41c9-b9d5-8e678168f689 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:05:57.463429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cb1a369d-c5bf-41c9-b9d5-8e678168f689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:57.567383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a369d-c5bf-41c9-b9d5-8e678168f689 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:05:57.646483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1564e89-1036-4e64-8c12-487c5c16a64d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:05:57.647197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1564e89-1036-4e64-8c12-487c5c16a64d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:57.647684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1564e89-1036-4e64-8c12-487c5c16a64d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:57.647684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1564e89-1036-4e64-8c12-487c5c16a64d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:57.649318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1564e89-1036-4e64-8c12-487c5c16a64d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:05:57.667922 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:05:57.761850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1564e89-1036-4e64-8c12-487c5c16a64d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:05:57.763362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1564e89-1036-4e64-8c12-487c5c16a64d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:05:57.781867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 652/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:05:59.901755 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0656a53-2c2e-4a7b-8173-c7e4c468dd0f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:59.902336 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0656a53-2c2e-4a7b-8173-c7e4c468dd0f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:59.902336 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0656a53-2c2e-4a7b-8173-c7e4c468dd0f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:59.904359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 677/1329] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:05:59.909469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024ecf7-baa6-479d-9f3d-5bd021a2d9df None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:05:59.909883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024ecf7-baa6-479d-9f3d-5bd021a2d9df None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:05:59.910031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024ecf7-baa6-479d-9f3d-5bd021a2d9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:05:59.976883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2024ecf7-baa6-479d-9f3d-5bd021a2d9df None None] MFA Rules not processed for user `6e889b4f5d384fef831673b867e27bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:05:59.978138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2024ecf7-baa6-479d-9f3d-5bd021a2d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:00.043759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:00.078573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 653/1330] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:05:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:00.116191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0656a53-2c2e-4a7b-8173-c7e4c468dd0f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:00.117742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a0656a53-2c2e-4a7b-8173-c7e4c468dd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:00.221071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0656a53-2c2e-4a7b-8173-c7e4c468dd0f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:00.241404 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:00.296363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26561d0f-bd00-4ffd-a0ac-2ac70308e7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:00.297018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26561d0f-bd00-4ffd-a0ac-2ac70308e7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:00.297306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26561d0f-bd00-4ffd-a0ac-2ac70308e7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:00.297585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26561d0f-bd00-4ffd-a0ac-2ac70308e7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:00.299394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-26561d0f-bd00-4ffd-a0ac-2ac70308e7e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:00.415219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26561d0f-bd00-4ffd-a0ac-2ac70308e7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:00.416860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26561d0f-bd00-4ffd-a0ac-2ac70308e7e9 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:00.439906 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 678/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:02.731753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05c15d-96eb-4c42-b536-20cfc98f4786 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:02.732280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05c15d-96eb-4c42-b536-20cfc98f4786 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:02.732398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05c15d-96eb-4c42-b536-20cfc98f4786 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:02.733697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 654/1332] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:02.739004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3d435-4410-47e6-838a-a67a65bdf4cb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:02.739440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3d435-4410-47e6-838a-a67a65bdf4cb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:02.739609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3d435-4410-47e6-838a-a67a65bdf4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:02.800275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:02.803576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b1d3d435-4410-47e6-838a-a67a65bdf4cb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:02.805483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b1d3d435-4410-47e6-838a-a67a65bdf4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:02.898949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 679/1333] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:02.935439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05c15d-96eb-4c42-b536-20cfc98f4786 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:02.936960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ab05c15d-96eb-4c42-b536-20cfc98f4786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:03.038244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05c15d-96eb-4c42-b536-20cfc98f4786 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:03.112926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b2d11-d8b7-43df-97d2-ac3af321af90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:03.113575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b2d11-d8b7-43df-97d2-ac3af321af90 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:03.113855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b2d11-d8b7-43df-97d2-ac3af321af90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:03.114128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b2d11-d8b7-43df-97d2-ac3af321af90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:03.115775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d26b2d11-d8b7-43df-97d2-ac3af321af90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:03.143294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:03.226532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b2d11-d8b7-43df-97d2-ac3af321af90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:03.227938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b2d11-d8b7-43df-97d2-ac3af321af90 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:03.245796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 655/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:05.389673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83296e-9edd-48e3-969a-f5c640f54f49 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:05.390241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83296e-9edd-48e3-969a-f5c640f54f49 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:05.390241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83296e-9edd-48e3-969a-f5c640f54f49 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:05.392040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 680/1335] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:05.397316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0c393-28de-4a74-9efc-8dbcb4939bb3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:05.397766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0c393-28de-4a74-9efc-8dbcb4939bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:05.397888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0c393-28de-4a74-9efc-8dbcb4939bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:05.462581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-08c0c393-28de-4a74-9efc-8dbcb4939bb3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:05.463809 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-08c0c393-28de-4a74-9efc-8dbcb4939bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:05.559931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:05.582049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 656/1336] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:05.622790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83296e-9edd-48e3-969a-f5c640f54f49 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:05.624336 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5c83296e-9edd-48e3-969a-f5c640f54f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:05.725246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83296e-9edd-48e3-969a-f5c640f54f49 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:05.752760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:05.797669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92028ceb-20b6-4fa5-b765-467ec42b6c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:05.798321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92028ceb-20b6-4fa5-b765-467ec42b6c7b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:05.798608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92028ceb-20b6-4fa5-b765-467ec42b6c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:05.798892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92028ceb-20b6-4fa5-b765-467ec42b6c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:05.800589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-92028ceb-20b6-4fa5-b765-467ec42b6c7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:05.916630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92028ceb-20b6-4fa5-b765-467ec42b6c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:05.918063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92028ceb-20b6-4fa5-b765-467ec42b6c7b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:05.935931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 681/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:06:08.358834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b81ca-cc7d-4c78-a102-26fce36ea22a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:08.359446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b81ca-cc7d-4c78-a102-26fce36ea22a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:08.359446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b81ca-cc7d-4c78-a102-26fce36ea22a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:08.360838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 657/1338] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:08.366303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306f62-7dcc-4448-b1d0-274e6c8de867 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:08.366733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306f62-7dcc-4448-b1d0-274e6c8de867 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:08.366874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306f62-7dcc-4448-b1d0-274e6c8de867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:08.430636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-37306f62-7dcc-4448-b1d0-274e6c8de867 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:08.431877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-37306f62-7dcc-4448-b1d0-274e6c8de867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:08.440682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:08.532065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 682/1339] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:08.569746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b81ca-cc7d-4c78-a102-26fce36ea22a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:08.571308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-531b81ca-cc7d-4c78-a102-26fce36ea22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:08.672262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b81ca-cc7d-4c78-a102-26fce36ea22a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:08.744081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579dc89-5975-4c94-9a4e-737bd12f2f6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:08.744734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579dc89-5975-4c94-9a4e-737bd12f2f6b service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:08.745021 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579dc89-5975-4c94-9a4e-737bd12f2f6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:08.745301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579dc89-5975-4c94-9a4e-737bd12f2f6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:08.746960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0579dc89-5975-4c94-9a4e-737bd12f2f6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:08.782442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:08.853240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579dc89-5975-4c94-9a4e-737bd12f2f6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:08.854677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579dc89-5975-4c94-9a4e-737bd12f2f6b service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:08.872631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 658/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:12.804405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4467ff-cf17-413f-b81e-f07ab522ef7c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:12.804947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4467ff-cf17-413f-b81e-f07ab522ef7c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:12.804947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4467ff-cf17-413f-b81e-f07ab522ef7c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:12.806333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 683/1341] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:12.812113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959a6bc-9cba-4c75-9e2b-5754f30e7e98 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:12.812536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959a6bc-9cba-4c75-9e2b-5754f30e7e98 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:12.812683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959a6bc-9cba-4c75-9e2b-5754f30e7e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:12.878250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8959a6bc-9cba-4c75-9e2b-5754f30e7e98 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:12.879476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8959a6bc-9cba-4c75-9e2b-5754f30e7e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:12.965373 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:12.976820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 659/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:06:13.016649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4467ff-cf17-413f-b81e-f07ab522ef7c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:13.018177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1b4467ff-cf17-413f-b81e-f07ab522ef7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:13.121197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4467ff-cf17-413f-b81e-f07ab522ef7c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:13.160220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:13.198057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df7926-1bb1-474c-a30c-b218d0b88e6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:13.198714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df7926-1bb1-474c-a30c-b218d0b88e6b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:13.199004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df7926-1bb1-474c-a30c-b218d0b88e6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:13.199318 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df7926-1bb1-474c-a30c-b218d0b88e6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:13.201114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-67df7926-1bb1-474c-a30c-b218d0b88e6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:13.310483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df7926-1bb1-474c-a30c-b218d0b88e6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:13.311975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df7926-1bb1-474c-a30c-b218d0b88e6b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:13.329950 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 684/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:15.940822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1f683-698c-4631-b01f-84ff984baaf8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:15.941363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1f683-698c-4631-b01f-84ff984baaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:15.941480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1f683-698c-4631-b01f-84ff984baaf8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:15.942810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 660/1344] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:15.948106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e641-3d2b-41e8-a927-623da0e1316a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:15.948516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e641-3d2b-41e8-a927-623da0e1316a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:15.948681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e641-3d2b-41e8-a927-623da0e1316a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:16.026707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-11c1e641-3d2b-41e8-a927-623da0e1316a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:16.028022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-11c1e641-3d2b-41e8-a927-623da0e1316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:16.047278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:16.133479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 685/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:16.173454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1f683-698c-4631-b01f-84ff984baaf8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:16.175386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e1c1f683-698c-4631-b01f-84ff984baaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:16.284538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1f683-698c-4631-b01f-84ff984baaf8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:16.357860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f3917-bba3-4ff6-8fde-2ff9392a16b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:16.358512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3917-bba3-4ff6-8fde-2ff9392a16b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:16.358794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3917-bba3-4ff6-8fde-2ff9392a16b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:16.359072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3917-bba3-4ff6-8fde-2ff9392a16b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:16.360755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c50f3917-bba3-4ff6-8fde-2ff9392a16b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:16.404090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:16.470108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f3917-bba3-4ff6-8fde-2ff9392a16b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:16.471585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f3917-bba3-4ff6-8fde-2ff9392a16b1 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:16.490521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 661/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:20.572741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f83358-ae5d-42e4-8de0-45fdabb4d26e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:20.573275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f83358-ae5d-42e4-8de0-45fdabb4d26e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:20.573369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f83358-ae5d-42e4-8de0-45fdabb4d26e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:20.574954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 686/1347] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:20.580117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690964f-ba57-44c6-ba13-914ad34eb599 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:20.580547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690964f-ba57-44c6-ba13-914ad34eb599 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:20.580689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690964f-ba57-44c6-ba13-914ad34eb599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:20.649024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0690964f-ba57-44c6-ba13-914ad34eb599 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:20.650252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0690964f-ba57-44c6-ba13-914ad34eb599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:20.750968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:20.755311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 662/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:20.797646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f83358-ae5d-42e4-8de0-45fdabb4d26e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:20.799176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e2f83358-ae5d-42e4-8de0-45fdabb4d26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:20.913395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f83358-ae5d-42e4-8de0-45fdabb4d26e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:20.961831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:20.989721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2403d-e707-4bef-9402-4f689b244bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:20.990304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2403d-e707-4bef-9402-4f689b244bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:20.990595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2403d-e707-4bef-9402-4f689b244bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:20.990875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2403d-e707-4bef-9402-4f689b244bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:20.992583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-92d2403d-e707-4bef-9402-4f689b244bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:21.115212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2403d-e707-4bef-9402-4f689b244bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:21.116704 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2403d-e707-4bef-9402-4f689b244bf1 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:21.136109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 687/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:26.104408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f0db7-defd-4259-8555-8044a4acb7fb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:26.104969 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f0db7-defd-4259-8555-8044a4acb7fb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:26.105221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f0db7-defd-4259-8555-8044a4acb7fb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:26.106921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 663/1350] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:06:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:26.111935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0359c8-4dee-45aa-9bb8-a4f78f856cca None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:26.112365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0359c8-4dee-45aa-9bb8-a4f78f856cca None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:26.112698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0359c8-4dee-45aa-9bb8-a4f78f856cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:26.237538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8e0359c8-4dee-45aa-9bb8-a4f78f856cca None None] MFA Rules not processed for user `88ecd4c645174d28930a44b93cfee9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:26.238657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8e0359c8-4dee-45aa-9bb8-a4f78f856cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:26.295698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:26.350488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 688/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:06:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:30.450003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a59d1-2e1d-4071-bd52-1e68223826be None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:30.450549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a59d1-2e1d-4071-bd52-1e68223826be None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:30.450549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a59d1-2e1d-4071-bd52-1e68223826be None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:30.451955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 664/1352] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:30.457320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9553a5-7482-4cf0-9505-c8bfdcb52341 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:30.458225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9553a5-7482-4cf0-9505-c8bfdcb52341 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:30.458875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9553a5-7482-4cf0-9505-c8bfdcb52341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:30.563105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-df9553a5-7482-4cf0-9505-c8bfdcb52341 None None] MFA Rules not processed for user `88ecd4c645174d28930a44b93cfee9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:30.564719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-df9553a5-7482-4cf0-9505-c8bfdcb52341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:30.649226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:30.693181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 689/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:06:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:32.014344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea78e34-efea-4a69-97f5-beb0e1437cdb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:32.014904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea78e34-efea-4a69-97f5-beb0e1437cdb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:32.014904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea78e34-efea-4a69-97f5-beb0e1437cdb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:32.016289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 665/1354] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:32.021221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836da07-7c9d-4861-a753-394abe545ce5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:32.021651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836da07-7c9d-4861-a753-394abe545ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:32.021789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836da07-7c9d-4861-a753-394abe545ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:32.096073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1836da07-7c9d-4861-a753-394abe545ce5 None None] MFA Rules not processed for user `88ecd4c645174d28930a44b93cfee9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:32.097272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1836da07-7c9d-4861-a753-394abe545ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:32.205620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 690/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:06:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:35.661855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d5b2-5404-4d07-ae8d-8958ba4958f7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:35.662793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d5b2-5404-4d07-ae8d-8958ba4958f7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:35.663309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d5b2-5404-4d07-ae8d-8958ba4958f7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:35.664910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 666/1356] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:35.670933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf0a11-f220-4d54-8ee7-04dcfee6fc2a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:35.671835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf0a11-f220-4d54-8ee7-04dcfee6fc2a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:35.672322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf0a11-f220-4d54-8ee7-04dcfee6fc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:35.713659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:35.748042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5bdf0a11-f220-4d54-8ee7-04dcfee6fc2a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:35.749237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5bdf0a11-f220-4d54-8ee7-04dcfee6fc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:35.848163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 691/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:35.904544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4942321-34b0-4d0c-9667-fb51cb811bd8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:35.904836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4942321-34b0-4d0c-9667-fb51cb811bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:35.905120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4942321-34b0-4d0c-9667-fb51cb811bd8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:35.906763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 667/1358] 10.222.0.20 () {54 vars in 962 bytes} [Fri Jul 22 11:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:35.918815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab7dac-502d-4d88-bb62-64f4906a6af9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:35.919910 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab7dac-502d-4d88-bb62-64f4906a6af9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:35.920764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab7dac-502d-4d88-bb62-64f4906a6af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:36.006543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2aab7dac-502d-4d88-bb62-64f4906a6af9 None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:36.007782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2aab7dac-502d-4d88-bb62-64f4906a6af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:36.058957 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:36.131101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 692/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Jul 22 11:06:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:06:36.140002 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4942321-34b0-4d0c-9667-fb51cb811bd8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:36.141745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e4942321-34b0-4d0c-9667-fb51cb811bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:36.316478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4942321-34b0-4d0c-9667-fb51cb811bd8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:36.397526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:36.436298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ef304-ef02-41b4-b415-00e7e9ca07e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:36.437043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ef304-ef02-41b4-b415-00e7e9ca07e9 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:36.437401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ef304-ef02-41b4-b415-00e7e9ca07e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:36.437747 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ef304-ef02-41b4-b415-00e7e9ca07e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:36.439471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a63ef304-ef02-41b4-b415-00e7e9ca07e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:36.534496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ef304-ef02-41b4-b415-00e7e9ca07e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:36.536133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ef304-ef02-41b4-b415-00e7e9ca07e9 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:36.554730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 668/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:38.818953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664482d5-72e1-43de-91a4-85484cd8bf78 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:38.819529 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664482d5-72e1-43de-91a4-85484cd8bf78 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:38.819529 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664482d5-72e1-43de-91a4-85484cd8bf78 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:38.821982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 693/1361] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:38.826111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59cce-0b97-43c9-99b6-194d78903d6e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:38.826543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59cce-0b97-43c9-99b6-194d78903d6e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:38.826670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59cce-0b97-43c9-99b6-194d78903d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:38.893246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-64c59cce-0b97-43c9-99b6-194d78903d6e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:38.894446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-64c59cce-0b97-43c9-99b6-194d78903d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:38.932368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:38.996037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 669/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:39.041435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664482d5-72e1-43de-91a4-85484cd8bf78 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:39.044113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-664482d5-72e1-43de-91a4-85484cd8bf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:39.222083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664482d5-72e1-43de-91a4-85484cd8bf78 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:39.233034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:39.385486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489568c-dc4c-4f51-9a9d-4097cc7f802d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:39.386287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489568c-dc4c-4f51-9a9d-4097cc7f802d service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:39.386643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489568c-dc4c-4f51-9a9d-4097cc7f802d service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:39.386995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489568c-dc4c-4f51-9a9d-4097cc7f802d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:39.388797 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c489568c-dc4c-4f51-9a9d-4097cc7f802d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:39.487609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489568c-dc4c-4f51-9a9d-4097cc7f802d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:39.489220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489568c-dc4c-4f51-9a9d-4097cc7f802d service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:39.508434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 694/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:41.896454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff631f9-1dc9-4fe6-b89d-8f30887406e4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:41.896982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff631f9-1dc9-4fe6-b89d-8f30887406e4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:41.897102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff631f9-1dc9-4fe6-b89d-8f30887406e4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:41.898680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 670/1364] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:41.904179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90b9bd-ab82-45b1-a5ca-13ee7d311312 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:41.904666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90b9bd-ab82-45b1-a5ca-13ee7d311312 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:41.904949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90b9bd-ab82-45b1-a5ca-13ee7d311312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:41.924061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:41.981218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ef90b9bd-ab82-45b1-a5ca-13ee7d311312 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:41.982583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ef90b9bd-ab82-45b1-a5ca-13ee7d311312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:42.092005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 695/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:06:42.138155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d8fd7-4d2e-479f-b117-677c53da06a2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:42.138578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d8fd7-4d2e-479f-b117-677c53da06a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:42.139121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d8fd7-4d2e-479f-b117-677c53da06a2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:42.140884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 671/1366] 10.222.0.20 () {54 vars in 962 bytes} [Fri Jul 22 11:06:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:42.146284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6d60b-6d01-4c15-9d9c-39c2cbd29b94 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:42.146805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6d60b-6d01-4c15-9d9c-39c2cbd29b94 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:42.147101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6d60b-6d01-4c15-9d9c-39c2cbd29b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:42.231381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-abe6d60b-6d01-4c15-9d9c-39c2cbd29b94 None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:42.232671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-abe6d60b-6d01-4c15-9d9c-39c2cbd29b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:42.268407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:42.359510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 696/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Jul 22 11:06:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:42.367752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d8fd7-4d2e-479f-b117-677c53da06a2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:42.369389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-902d8fd7-4d2e-479f-b117-677c53da06a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:42.534105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d8fd7-4d2e-479f-b117-677c53da06a2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:42.555062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:42.661915 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f976c-e93e-430e-b61b-719aaa99a4c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:42.662569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f976c-e93e-430e-b61b-719aaa99a4c8 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:42.662854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f976c-e93e-430e-b61b-719aaa99a4c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:42.663129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f976c-e93e-430e-b61b-719aaa99a4c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:42.664796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f3f976c-e93e-430e-b61b-719aaa99a4c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:42.755916 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f976c-e93e-430e-b61b-719aaa99a4c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:42.757451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f976c-e93e-430e-b61b-719aaa99a4c8 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:42.778940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 672/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:43.749535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8470b-2117-427c-8a92-ae01df1dc57f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:43.750033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8470b-2117-427c-8a92-ae01df1dc57f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:43.750033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8470b-2117-427c-8a92-ae01df1dc57f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:43.751789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 697/1369] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:43.757009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a468d6-b64f-48eb-bb61-4e8f11d2e6b1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:43.757482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a468d6-b64f-48eb-bb61-4e8f11d2e6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:43.757588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a468d6-b64f-48eb-bb61-4e8f11d2e6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:43.793885 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:43.830250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a8a468d6-b64f-48eb-bb61-4e8f11d2e6b1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:43.831547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a8a468d6-b64f-48eb-bb61-4e8f11d2e6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:43.934782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 673/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:44.285397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c3f87-5d68-44dc-b14b-e1137f010fd8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:44.285692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c3f87-5d68-44dc-b14b-e1137f010fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:44.285976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c3f87-5d68-44dc-b14b-e1137f010fd8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:44.288038 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 698/1371] 10.222.0.20 () {54 vars in 964 bytes} [Fri Jul 22 11:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:44.292576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47861778-d8d2-4d27-8bad-e150e1eafb68 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:44.292871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47861778-d8d2-4d27-8bad-e150e1eafb68 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:44.293153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47861778-d8d2-4d27-8bad-e150e1eafb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:44.373695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-47861778-d8d2-4d27-8bad-e150e1eafb68 None None] MFA Rules not processed for user `5b04cb4117974f8eb9bbf3a41c48a11a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:44.374926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-47861778-d8d2-4d27-8bad-e150e1eafb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:44.426119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:44.463382 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 674/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Fri Jul 22 11:06:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:44.470919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c3f87-5d68-44dc-b14b-e1137f010fd8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:44.472922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5b8c3f87-5d68-44dc-b14b-e1137f010fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:44.587068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c3f87-5d68-44dc-b14b-e1137f010fd8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:44.640468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:44.678871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571d77f-06d6-4357-9480-48f8dfaed5fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:44.679564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571d77f-06d6-4357-9480-48f8dfaed5fe service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:44.679856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571d77f-06d6-4357-9480-48f8dfaed5fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:44.680137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571d77f-06d6-4357-9480-48f8dfaed5fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:44.681778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c571d77f-06d6-4357-9480-48f8dfaed5fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:44.804656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571d77f-06d6-4357-9480-48f8dfaed5fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:44.806231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571d77f-06d6-4357-9480-48f8dfaed5fe service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:44.835861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 699/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:44.892456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a445e6c-7e66-4494-b1d3-19808a377440 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:44.892892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a445e6c-7e66-4494-b1d3-19808a377440 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:44.893412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a445e6c-7e66-4494-b1d3-19808a377440 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:44.895438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 675/1374] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:44.900811 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1eab3-d483-48e8-ae2f-3da00c01fff7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:44.901048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1eab3-d483-48e8-ae2f-3da00c01fff7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:44.901365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1eab3-d483-48e8-ae2f-3da00c01fff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:44.966903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c2c1eab3-d483-48e8-ae2f-3da00c01fff7 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:44.968172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c2c1eab3-d483-48e8-ae2f-3da00c01fff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:44.990577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:45.095425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 700/1375] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:45.136416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a445e6c-7e66-4494-b1d3-19808a377440 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:45.138126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4a445e6c-7e66-4494-b1d3-19808a377440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:45.293533 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a445e6c-7e66-4494-b1d3-19808a377440 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:45.347070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:45.408483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23dc83-499e-4f50-97c8-01879ec4329f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:45.409137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23dc83-499e-4f50-97c8-01879ec4329f service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:45.409426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23dc83-499e-4f50-97c8-01879ec4329f service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:45.409705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23dc83-499e-4f50-97c8-01879ec4329f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:45.411381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5b23dc83-499e-4f50-97c8-01879ec4329f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:45.502696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23dc83-499e-4f50-97c8-01879ec4329f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:45.504332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23dc83-499e-4f50-97c8-01879ec4329f service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:45.523913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 676/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:47.576164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478746af-ba73-4a7b-af8a-4c82ac8b7197 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:47.576692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478746af-ba73-4a7b-af8a-4c82ac8b7197 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:47.577172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478746af-ba73-4a7b-af8a-4c82ac8b7197 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:47.578702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 701/1377] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:47.584097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40847f-9fcd-4dcb-ad54-fc61689c8b47 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:47.584556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40847f-9fcd-4dcb-ad54-fc61689c8b47 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:47.584656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40847f-9fcd-4dcb-ad54-fc61689c8b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:47.656720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eb40847f-9fcd-4dcb-ad54-fc61689c8b47 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:47.657941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eb40847f-9fcd-4dcb-ad54-fc61689c8b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:47.664624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:47.762530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 677/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:06:47.816335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478746af-ba73-4a7b-af8a-4c82ac8b7197 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:47.818213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-478746af-ba73-4a7b-af8a-4c82ac8b7197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:47.968723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478746af-ba73-4a7b-af8a-4c82ac8b7197 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:47.972186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:48.085160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e63bd-191f-412f-9469-8801b17ca0ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:48.085810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e63bd-191f-412f-9469-8801b17ca0ae service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:48.086097 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e63bd-191f-412f-9469-8801b17ca0ae service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:48.086378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e63bd-191f-412f-9469-8801b17ca0ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:48.088034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7a5e63bd-191f-412f-9469-8801b17ca0ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:48.193404 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e63bd-191f-412f-9469-8801b17ca0ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:48.195037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e63bd-191f-412f-9469-8801b17ca0ae service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:48.213424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 702/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:50.309357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b612af-682b-46d4-a42c-85a0a43a90d3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:50.309896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b612af-682b-46d4-a42c-85a0a43a90d3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:50.310161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b612af-682b-46d4-a42c-85a0a43a90d3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:50.311849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 678/1380] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:50.317104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb89e34-7cbf-4432-af14-cbabe0f63af1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:50.317567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb89e34-7cbf-4432-af14-cbabe0f63af1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:50.317666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb89e34-7cbf-4432-af14-cbabe0f63af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:50.329322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:50.408597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2bb89e34-7cbf-4432-af14-cbabe0f63af1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:50.411628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2bb89e34-7cbf-4432-af14-cbabe0f63af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:50.534136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 703/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:50.572006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b612af-682b-46d4-a42c-85a0a43a90d3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:50.574471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-73b612af-682b-46d4-a42c-85a0a43a90d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:50.740279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:50.752088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b612af-682b-46d4-a42c-85a0a43a90d3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:50.878637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc40199-2dd4-4686-ad9f-e3364b559dc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:50.879434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc40199-2dd4-4686-ad9f-e3364b559dc1 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:50.879790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc40199-2dd4-4686-ad9f-e3364b559dc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:50.880232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc40199-2dd4-4686-ad9f-e3364b559dc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:50.882409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-abc40199-2dd4-4686-ad9f-e3364b559dc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:50.982467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc40199-2dd4-4686-ad9f-e3364b559dc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:50.984211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc40199-2dd4-4686-ad9f-e3364b559dc1 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:50.996733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:51.006318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 679/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:53.183287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65323e-254f-4122-b0e4-568b030582ff None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:53.184028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65323e-254f-4122-b0e4-568b030582ff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:53.184028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65323e-254f-4122-b0e4-568b030582ff None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:53.185240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 704/1383] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:06:53.191040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d1881-7d9a-4c3b-aaf5-614aa68011e6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:53.191998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d1881-7d9a-4c3b-aaf5-614aa68011e6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:53.191998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d1881-7d9a-4c3b-aaf5-614aa68011e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:53.277969 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2c1d1881-7d9a-4c3b-aaf5-614aa68011e6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:53.279204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c1d1881-7d9a-4c3b-aaf5-614aa68011e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:53.388927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 680/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:06:53.443824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65323e-254f-4122-b0e4-568b030582ff None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:53.445399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6b65323e-254f-4122-b0e4-568b030582ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:53.548776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:53.613484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65323e-254f-4122-b0e4-568b030582ff None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:53.724914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ebf60-40e8-4dce-8b74-29a5135bb173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:53.725484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebf60-40e8-4dce-8b74-29a5135bb173 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:53.725768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebf60-40e8-4dce-8b74-29a5135bb173 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:53.726042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebf60-40e8-4dce-8b74-29a5135bb173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:53.727702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-441ebf60-40e8-4dce-8b74-29a5135bb173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:53.780212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:53.821963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ebf60-40e8-4dce-8b74-29a5135bb173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:53.824198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ebf60-40e8-4dce-8b74-29a5135bb173 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:53.847783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 705/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:55.935482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996b80-b783-43df-a70e-d40bc348728b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:55.936033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996b80-b783-43df-a70e-d40bc348728b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:55.936197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996b80-b783-43df-a70e-d40bc348728b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:55.938098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 681/1386] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:55.943484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25126309-57ad-4ec9-9008-51ccc98615ab None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:55.943896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25126309-57ad-4ec9-9008-51ccc98615ab None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:55.944130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25126309-57ad-4ec9-9008-51ccc98615ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:56.022324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-25126309-57ad-4ec9-9008-51ccc98615ab None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:56.023960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-25126309-57ad-4ec9-9008-51ccc98615ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:56.127864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 706/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:56.175888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996b80-b783-43df-a70e-d40bc348728b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:56.177433 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-26996b80-b783-43df-a70e-d40bc348728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:56.259435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:56.338436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996b80-b783-43df-a70e-d40bc348728b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:56.455078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29e11d-57b9-4272-be75-1706f3badf93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:56.455796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29e11d-57b9-4272-be75-1706f3badf93 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:56.456958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29e11d-57b9-4272-be75-1706f3badf93 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:56.457252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29e11d-57b9-4272-be75-1706f3badf93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:56.459951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-df29e11d-57b9-4272-be75-1706f3badf93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:56.499573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:56.558933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29e11d-57b9-4272-be75-1706f3badf93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:56.562221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29e11d-57b9-4272-be75-1706f3badf93 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:56.595950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 682/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:59.039973 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01510f9f-9d48-4853-b173-057a5815046b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:59.040531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01510f9f-9d48-4853-b173-057a5815046b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:59.040531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01510f9f-9d48-4853-b173-057a5815046b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:59.041938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 707/1389] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:59.047430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3645913-574d-41c0-9155-edbf9e4064d7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:59.047863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3645913-574d-41c0-9155-edbf9e4064d7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:59.048140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3645913-574d-41c0-9155-edbf9e4064d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:59.112396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e3645913-574d-41c0-9155-edbf9e4064d7 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:59.113606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e3645913-574d-41c0-9155-edbf9e4064d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:59.208845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 683/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:06:59.255738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01510f9f-9d48-4853-b173-057a5815046b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:06:59.257248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-01510f9f-9d48-4853-b173-057a5815046b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:59.288522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:59.408837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01510f9f-9d48-4853-b173-057a5815046b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:06:59.511984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:06:59.523929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019864f2-bdc4-4a6f-ad4c-40b0ce37db62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:06:59.524610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019864f2-bdc4-4a6f-ad4c-40b0ce37db62 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:59.524901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019864f2-bdc4-4a6f-ad4c-40b0ce37db62 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:59.525186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019864f2-bdc4-4a6f-ad4c-40b0ce37db62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:59.526854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-019864f2-bdc4-4a6f-ad4c-40b0ce37db62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:06:59.624358 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019864f2-bdc4-4a6f-ad4c-40b0ce37db62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:06:59.625989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019864f2-bdc4-4a6f-ad4c-40b0ce37db62 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:06:59.644592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 708/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:06:59.898741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8420cf9-bfb8-4815-976c-d43b915935da None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:59.899172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8420cf9-bfb8-4815-976c-d43b915935da None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:59.899506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8420cf9-bfb8-4815-976c-d43b915935da None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:59.900910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 684/1392] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:06:59.906567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b80f58-2a97-44f2-be20-5cc049743654 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:06:59.906936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b80f58-2a97-44f2-be20-5cc049743654 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:06:59.907226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b80f58-2a97-44f2-be20-5cc049743654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:06:59.972231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f5b80f58-2a97-44f2-be20-5cc049743654 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:06:59.973359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f5b80f58-2a97-44f2-be20-5cc049743654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:00.031968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:00.070779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 709/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:00.127900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bc00a-3c0a-455e-985e-df3a1a93f0db None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:00.128376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bc00a-3c0a-455e-985e-df3a1a93f0db None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:00.128475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bc00a-3c0a-455e-985e-df3a1a93f0db None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:00.129859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 685/1394] 10.222.0.20 () {54 vars in 966 bytes} [Fri Jul 22 11:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:00.134901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c20abc-af62-4a4b-ba15-ac93e67b6e8f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:00.135237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c20abc-af62-4a4b-ba15-ac93e67b6e8f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:00.135618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c20abc-af62-4a4b-ba15-ac93e67b6e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:00.209369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e3c20abc-af62-4a4b-ba15-ac93e67b6e8f None None] MFA Rules not processed for user `92fc25f9ad7c49c48a909d2d92ea0a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:00.210558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e3c20abc-af62-4a4b-ba15-ac93e67b6e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:00.287509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 710/1395] 10.222.0.20 () {58 vars in 1063 bytes} [Fri Jul 22 11:07:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:00.295175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bc00a-3c0a-455e-985e-df3a1a93f0db None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:00.296729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-437bc00a-3c0a-455e-985e-df3a1a93f0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:00.310050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:00.401753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bc00a-3c0a-455e-985e-df3a1a93f0db None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:00.476024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656059f-5493-4d3b-b97a-93e34d6318c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92fc25f9ad7c49c48a909d2d92ea0a66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:00.476675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656059f-5493-4d3b-b97a-93e34d6318c8 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:00.476962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656059f-5493-4d3b-b97a-93e34d6318c8 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:00.477243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656059f-5493-4d3b-b97a-93e34d6318c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:00.478895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e656059f-5493-4d3b-b97a-93e34d6318c8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:00.545887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:00.588240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656059f-5493-4d3b-b97a-93e34d6318c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:00.589687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656059f-5493-4d3b-b97a-93e34d6318c8 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:00.609207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 686/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:01.317991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8b192-4fbf-4833-82db-2ccd9722edf2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:01.318465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8b192-4fbf-4833-82db-2ccd9722edf2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:01.318598 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8b192-4fbf-4833-82db-2ccd9722edf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:01.383895 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5bc8b192-4fbf-4833-82db-2ccd9722edf2 None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:01.385633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5bc8b192-4fbf-4833-82db-2ccd9722edf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:01.394071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:01.478547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 711/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:01.521111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ace3-d123-46b6-ae5a-7df1403eb72c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:01.521599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ace3-d123-46b6-ae5a-7df1403eb72c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:01.521759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ace3-d123-46b6-ae5a-7df1403eb72c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:01.523186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 687/1398] 10.222.0.20 () {54 vars in 943 bytes} [Fri Jul 22 11:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:01.530240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491e291-4ee8-42f3-bb98-b70d5d1ef20a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:01.530608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491e291-4ee8-42f3-bb98-b70d5d1ef20a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:01.530981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491e291-4ee8-42f3-bb98-b70d5d1ef20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:01.729292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a491e291-4ee8-42f3-bb98-b70d5d1ef20a None None] MFA Rules not processed for user `e7935e85cc3f4ed1ad5b01ba892e6e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:01.730503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a491e291-4ee8-42f3-bb98-b70d5d1ef20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:01.753078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7261a3a-6d7d-4bf9-a535-31f968bf048d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:01.753365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7261a3a-6d7d-4bf9-a535-31f968bf048d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:01.753662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7261a3a-6d7d-4bf9-a535-31f968bf048d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:01.755156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 688/1399] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:07:01.759822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a9184-dd84-413a-bc0f-98728b8451a0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:01.760729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a9184-dd84-413a-bc0f-98728b8451a0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:01.761016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a9184-dd84-413a-bc0f-98728b8451a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:01.820993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:01.829701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ce3a9184-dd84-413a-bc0f-98728b8451a0 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:01.830924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce3a9184-dd84-413a-bc0f-98728b8451a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:01.845097 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 712/1400] 10.222.0.20 () {58 vars in 1040 bytes} [Fri Jul 22 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:01.854479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491e291-4ee8-42f3-bb98-b70d5d1ef20a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:01.864342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491e291-4ee8-42f3-bb98-b70d5d1ef20a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:01.933341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 689/1401] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:07:01.975549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05fef5-f67a-456f-99ff-ccd648b0c561 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:01.976201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05fef5-f67a-456f-99ff-ccd648b0c561 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:01.976484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05fef5-f67a-456f-99ff-ccd648b0c561 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:01.976760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05fef5-f67a-456f-99ff-ccd648b0c561 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:01.978174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a9184-dd84-413a-bc0f-98728b8451a0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:01.978424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3e05fef5-f67a-456f-99ff-ccd648b0c561 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:01.979952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce3a9184-dd84-413a-bc0f-98728b8451a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:02.007369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:02.062737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05fef5-f67a-456f-99ff-ccd648b0c561 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:02.064373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05fef5-f67a-456f-99ff-ccd648b0c561 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:02.072685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 713/1402] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:07:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:07:02.097615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a9184-dd84-413a-bc0f-98728b8451a0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:02.226670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6adeb-66ca-4c8e-bfb1-451951323f9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:02.227492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6adeb-66ca-4c8e-bfb1-451951323f9e service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:02.227845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6adeb-66ca-4c8e-bfb1-451951323f9e service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:02.228192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6adeb-66ca-4c8e-bfb1-451951323f9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:02.230612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-09f6adeb-66ca-4c8e-bfb1-451951323f9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:02.250649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:02.329143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6adeb-66ca-4c8e-bfb1-451951323f9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:02.330941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6adeb-66ca-4c8e-bfb1-451951323f9e service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:02.353215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 690/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:07:04.413918 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0efcb-09d7-4ebc-a648-b5ff742886f1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:04.414453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0efcb-09d7-4ebc-a648-b5ff742886f1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:04.414453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0efcb-09d7-4ebc-a648-b5ff742886f1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:04.415851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 714/1404] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:04.421310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548531c-dc80-4d6b-8bd9-c3169a43bb7e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:04.421703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548531c-dc80-4d6b-8bd9-c3169a43bb7e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:04.421876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548531c-dc80-4d6b-8bd9-c3169a43bb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:04.500090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c548531c-dc80-4d6b-8bd9-c3169a43bb7e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:04.501890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c548531c-dc80-4d6b-8bd9-c3169a43bb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:04.604714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:04.611903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 691/1405] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:04.657849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0efcb-09d7-4ebc-a648-b5ff742886f1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:04.659427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a7a0efcb-09d7-4ebc-a648-b5ff742886f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:04.670452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:04.826501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0efcb-09d7-4ebc-a648-b5ff742886f1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:04.915723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:04.956771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dca16-e89b-45a5-ba6a-5993dd480bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:04.957338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dca16-e89b-45a5-ba6a-5993dd480bdd service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:04.957622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dca16-e89b-45a5-ba6a-5993dd480bdd service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:04.957895 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dca16-e89b-45a5-ba6a-5993dd480bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:04.959588 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-373dca16-e89b-45a5-ba6a-5993dd480bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:05.059581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dca16-e89b-45a5-ba6a-5993dd480bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:05.061563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dca16-e89b-45a5-ba6a-5993dd480bdd service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:05.080414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 715/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:07.219672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9dc0-828a-4fbe-b102-789cdae66e5f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:07.219672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9dc0-828a-4fbe-b102-789cdae66e5f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:07.219672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9dc0-828a-4fbe-b102-789cdae66e5f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:07.225201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 692/1407] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:07:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:07.226708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b950b5-1756-4bb4-a787-2449a619d453 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:07.227180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b950b5-1756-4bb4-a787-2449a619d453 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:07.227382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b950b5-1756-4bb4-a787-2449a619d453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:07.296359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-56b950b5-1756-4bb4-a787-2449a619d453 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:07.297589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-56b950b5-1756-4bb4-a787-2449a619d453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:07.348366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:07.408680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 716/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:07:07.455838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9dc0-828a-4fbe-b102-789cdae66e5f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:07.457626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7b1d9dc0-828a-4fbe-b102-789cdae66e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:07.638642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9dc0-828a-4fbe-b102-789cdae66e5f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:07.724083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:07.785172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3eb21b-a41d-467f-b93a-2bfe76ecd625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:07.785934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eb21b-a41d-467f-b93a-2bfe76ecd625 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:07.786297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eb21b-a41d-467f-b93a-2bfe76ecd625 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:07.786665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eb21b-a41d-467f-b93a-2bfe76ecd625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:07.788455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0a3eb21b-a41d-467f-b93a-2bfe76ecd625 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:07.896905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3eb21b-a41d-467f-b93a-2bfe76ecd625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:07.898484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3eb21b-a41d-467f-b93a-2bfe76ecd625 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:07.920765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 693/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:17.406611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=23 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:17.918419 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=64 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:24.500786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b82c7d-cb93-4cbb-b7f0-d2ab27a8c2b1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:24.501369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b82c7d-cb93-4cbb-b7f0-d2ab27a8c2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:24.501481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b82c7d-cb93-4cbb-b7f0-d2ab27a8c2b1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:24.502871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 717/1410] 10.222.0.20 () {54 vars in 918 bytes} [Fri Jul 22 11:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:24.508063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4cfac-e8c2-4048-8d5e-17488997accc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:24.508479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4cfac-e8c2-4048-8d5e-17488997accc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:24.508630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4cfac-e8c2-4048-8d5e-17488997accc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:24.581313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-44b4cfac-e8c2-4048-8d5e-17488997accc None None] MFA Rules not processed for user `88ecd4c645174d28930a44b93cfee9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:24.582539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-44b4cfac-e8c2-4048-8d5e-17488997accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:24.695066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 694/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Jul 22 11:07:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:24.709678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57efb51-5398-4953-b41e-10745a144808 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:24.709975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57efb51-5398-4953-b41e-10745a144808 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:24.710260 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57efb51-5398-4953-b41e-10745a144808 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:24.711686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 718/1412] 10.222.0.20 () {54 vars in 918 bytes} [Fri Jul 22 11:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:24.716368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83593-979a-4b86-8fa7-ffa929a102a4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:24.716660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83593-979a-4b86-8fa7-ffa929a102a4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:24.716940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83593-979a-4b86-8fa7-ffa929a102a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:24.786710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:24.789586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1ca83593-979a-4b86-8fa7-ffa929a102a4 None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:24.791984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1ca83593-979a-4b86-8fa7-ffa929a102a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:24.905288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 695/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Jul 22 11:07:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:27.397280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd27de-e9d0-4097-b81a-01554c847727 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:27.397862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd27de-e9d0-4097-b81a-01554c847727 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:27.397862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd27de-e9d0-4097-b81a-01554c847727 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:27.399235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:27.404555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a22ef-a66b-418b-9cc9-f2d719d9bdd5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:27.405039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a22ef-a66b-418b-9cc9-f2d719d9bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:27.405144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a22ef-a66b-418b-9cc9-f2d719d9bdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:27.470763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5d4a22ef-a66b-418b-9cc9-f2d719d9bdd5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:27.472014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5d4a22ef-a66b-418b-9cc9-f2d719d9bdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:27.526786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:27.573477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:27.616459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd27de-e9d0-4097-b81a-01554c847727 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:27.618137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-69fd27de-e9d0-4097-b81a-01554c847727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:27.763139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd27de-e9d0-4097-b81a-01554c847727 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:27.833596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:27.874149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc970a3-48cf-4d4c-aba8-38808cd3d861 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:27.874795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc970a3-48cf-4d4c-aba8-38808cd3d861 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:27.875078 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc970a3-48cf-4d4c-aba8-38808cd3d861 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:27.875383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc970a3-48cf-4d4c-aba8-38808cd3d861 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:27.876997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3dc970a3-48cf-4d4c-aba8-38808cd3d861 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:27.963316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc970a3-48cf-4d4c-aba8-38808cd3d861 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:27.964891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc970a3-48cf-4d4c-aba8-38808cd3d861 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:27.982689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 720/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:28.876256 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa50633-57c2-44af-b9e8-0250ac798e73 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:28.876793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa50633-57c2-44af-b9e8-0250ac798e73 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:28.876793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa50633-57c2-44af-b9e8-0250ac798e73 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:28.878114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:28.883001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d0d8e-2dcf-4c42-81fb-341e3fbb71d3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:28.883450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d0d8e-2dcf-4c42-81fb-341e3fbb71d3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:28.883590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d0d8e-2dcf-4c42-81fb-341e3fbb71d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:28.955523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-aa9d0d8e-2dcf-4c42-81fb-341e3fbb71d3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:28.956710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aa9d0d8e-2dcf-4c42-81fb-341e3fbb71d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:28.983815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:29.053175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:29.202172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa50633-57c2-44af-b9e8-0250ac798e73 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:29.203698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-efa50633-57c2-44af-b9e8-0250ac798e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:29.347826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa50633-57c2-44af-b9e8-0250ac798e73 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:29.386693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:29.457454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f3503-a370-4f88-8b08-688112203128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:29.458098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f3503-a370-4f88-8b08-688112203128 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:29.458379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f3503-a370-4f88-8b08-688112203128 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:29.458653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f3503-a370-4f88-8b08-688112203128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:29.460315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a83f3503-a370-4f88-8b08-688112203128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:29.548257 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f3503-a370-4f88-8b08-688112203128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:29.549853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f3503-a370-4f88-8b08-688112203128 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:29.567186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 698/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:30.455937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d843dc0-f588-4fca-a0e8-241c6128e8c8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:30.456519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d843dc0-f588-4fca-a0e8-241c6128e8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:30.456519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d843dc0-f588-4fca-a0e8-241c6128e8c8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:30.457887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:30.462817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22181fd2-e824-4509-9e43-a677415bdd0e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:30.463290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22181fd2-e824-4509-9e43-a677415bdd0e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:30.463425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22181fd2-e824-4509-9e43-a677415bdd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:30.517962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:30.539456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-22181fd2-e824-4509-9e43-a677415bdd0e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:30.540693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22181fd2-e824-4509-9e43-a677415bdd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:30.642867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:30.793402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d843dc0-f588-4fca-a0e8-241c6128e8c8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:30.794894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9d843dc0-f588-4fca-a0e8-241c6128e8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:30.936682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d843dc0-f588-4fca-a0e8-241c6128e8c8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:30.948324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:31.046311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d299c2-4a11-4e32-adc4-6d887115a85b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:31.046877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d299c2-4a11-4e32-adc4-6d887115a85b service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:31.047166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d299c2-4a11-4e32-adc4-6d887115a85b service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:31.047529 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d299c2-4a11-4e32-adc4-6d887115a85b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:31.049141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-59d299c2-4a11-4e32-adc4-6d887115a85b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:31.133840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d299c2-4a11-4e32-adc4-6d887115a85b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:31.135426 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d299c2-4a11-4e32-adc4-6d887115a85b service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:31.154055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 723/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:32.005062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfdacc-5cdc-4e41-af6a-18026c211d72 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:32.005633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfdacc-5cdc-4e41-af6a-18026c211d72 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:32.005633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfdacc-5cdc-4e41-af6a-18026c211d72 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:32.006991 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:32.012037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a58286-118a-4d49-9ed5-cdc891d54923 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:32.012431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a58286-118a-4d49-9ed5-cdc891d54923 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:32.012631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a58286-118a-4d49-9ed5-cdc891d54923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:32.030672 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:32.086172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-21a58286-118a-4d49-9ed5-cdc891d54923 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:32.087413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-21a58286-118a-4d49-9ed5-cdc891d54923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:32.181379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:32.329750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfdacc-5cdc-4e41-af6a-18026c211d72 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:32.331299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c0bfdacc-5cdc-4e41-af6a-18026c211d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:32.437860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:32.473449 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfdacc-5cdc-4e41-af6a-18026c211d72 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:32.584258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d08bb-edca-48be-9331-c5521f3d53a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:32.585665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d08bb-edca-48be-9331-c5521f3d53a7 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:32.585947 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d08bb-edca-48be-9331-c5521f3d53a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:32.586227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d08bb-edca-48be-9331-c5521f3d53a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:32.587911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-155d08bb-edca-48be-9331-c5521f3d53a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:32.665158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:32.678703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d08bb-edca-48be-9331-c5521f3d53a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:32.680341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d08bb-edca-48be-9331-c5521f3d53a7 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:32.698641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 701/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:33.552140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e9aec-22b5-4dc7-9743-bdee3009c354 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:33.552703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e9aec-22b5-4dc7-9743-bdee3009c354 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:33.552703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e9aec-22b5-4dc7-9743-bdee3009c354 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:33.554078 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:33.559729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9f57d-f5fa-48ba-8a1a-a6ba7baf1118 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:33.560154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9f57d-f5fa-48ba-8a1a-a6ba7baf1118 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:33.560281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9f57d-f5fa-48ba-8a1a-a6ba7baf1118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:33.636289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d3f9f57d-f5fa-48ba-8a1a-a6ba7baf1118 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:33.637497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d3f9f57d-f5fa-48ba-8a1a-a6ba7baf1118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:33.731250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:33.882662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e9aec-22b5-4dc7-9743-bdee3009c354 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:33.884208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9f3e9aec-22b5-4dc7-9743-bdee3009c354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:33.965937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:34.027689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e9aec-22b5-4dc7-9743-bdee3009c354 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:34.144955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429af2bd-444d-4952-a92c-564d4d953a98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:34.145599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429af2bd-444d-4952-a92c-564d4d953a98 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:34.145886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429af2bd-444d-4952-a92c-564d4d953a98 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:34.146163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429af2bd-444d-4952-a92c-564d4d953a98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:34.147849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-429af2bd-444d-4952-a92c-564d4d953a98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:34.194932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:34.235504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429af2bd-444d-4952-a92c-564d4d953a98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:34.237347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429af2bd-444d-4952-a92c-564d4d953a98 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:34.254582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 726/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:35.052116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04dab0-fe8a-4525-918a-075187137be6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:35.052662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04dab0-fe8a-4525-918a-075187137be6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:35.052662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04dab0-fe8a-4525-918a-075187137be6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:35.054072 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:35.059615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131fac3-3550-4c71-9505-38548901023d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:35.060019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131fac3-3550-4c71-9505-38548901023d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:35.060174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131fac3-3550-4c71-9505-38548901023d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:35.132517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f131fac3-3550-4c71-9505-38548901023d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:35.133714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f131fac3-3550-4c71-9505-38548901023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:35.226872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:35.376591 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04dab0-fe8a-4525-918a-075187137be6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:35.378221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cc04dab0-fe8a-4525-918a-075187137be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:35.425593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:35.521192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04dab0-fe8a-4525-918a-075187137be6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:35.632887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5381b-7b96-4703-8b5f-23326732192f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:35.633465 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5381b-7b96-4703-8b5f-23326732192f service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:35.633753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5381b-7b96-4703-8b5f-23326732192f service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:35.634032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5381b-7b96-4703-8b5f-23326732192f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:35.636231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2ad5381b-7b96-4703-8b5f-23326732192f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:35.643023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:35.724378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5381b-7b96-4703-8b5f-23326732192f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:35.726020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5381b-7b96-4703-8b5f-23326732192f service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:35.743868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 704/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:36.557998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741d1ee-7462-410c-88ef-dfd1a9147613 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:36.558576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741d1ee-7462-410c-88ef-dfd1a9147613 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:36.558576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741d1ee-7462-410c-88ef-dfd1a9147613 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:36.559991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:36.564937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62294f3a-5386-497e-a42e-3182831d6c88 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:36.565354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62294f3a-5386-497e-a42e-3182831d6c88 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:36.565538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62294f3a-5386-497e-a42e-3182831d6c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:36.642932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-62294f3a-5386-497e-a42e-3182831d6c88 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:36.644171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-62294f3a-5386-497e-a42e-3182831d6c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:36.709190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:36.738926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:36.888243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741d1ee-7462-410c-88ef-dfd1a9147613 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:36.889671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8741d1ee-7462-410c-88ef-dfd1a9147613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:36.906904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:37.040368 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741d1ee-7462-410c-88ef-dfd1a9147613 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:37.126075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:37.152868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f25c3-3c32-4df3-8bdc-5705d8ad9d6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:37.153451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f25c3-3c32-4df3-8bdc-5705d8ad9d6a service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:37.153737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f25c3-3c32-4df3-8bdc-5705d8ad9d6a service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:37.154017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f25c3-3c32-4df3-8bdc-5705d8ad9d6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:37.155691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-389f25c3-3c32-4df3-8bdc-5705d8ad9d6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:37.243785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f25c3-3c32-4df3-8bdc-5705d8ad9d6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:37.245970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f25c3-3c32-4df3-8bdc-5705d8ad9d6a service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:37.263411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 729/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:38.067268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6c54d-8491-45b6-a333-24956537899f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:38.067844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6c54d-8491-45b6-a333-24956537899f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:38.067967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6c54d-8491-45b6-a333-24956537899f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:38.069294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:38.074744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6c4da-ad99-40cf-89e9-9e97e97f4b07 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:38.075182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6c4da-ad99-40cf-89e9-9e97e97f4b07 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:38.075317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6c4da-ad99-40cf-89e9-9e97e97f4b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:38.148729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9dd6c4da-ad99-40cf-89e9-9e97e97f4b07 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:38.149935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9dd6c4da-ad99-40cf-89e9-9e97e97f4b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:38.190863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:38.249343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:38.399000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6c54d-8491-45b6-a333-24956537899f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:38.400506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-33d6c54d-8491-45b6-a333-24956537899f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:38.547589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6c54d-8491-45b6-a333-24956537899f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:38.602187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:38.660241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d498c-0425-4448-bd5c-2ff012063b9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:38.660810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d498c-0425-4448-bd5c-2ff012063b9f service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:38.661096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d498c-0425-4448-bd5c-2ff012063b9f service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:38.661373 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d498c-0425-4448-bd5c-2ff012063b9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:38.663003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e45d498c-0425-4448-bd5c-2ff012063b9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:38.759651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d498c-0425-4448-bd5c-2ff012063b9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:38.761272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d498c-0425-4448-bd5c-2ff012063b9f service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:38.780444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 707/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:39.638553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af13d5d-92b5-4b77-a990-ed052f65e676 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:39.639140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af13d5d-92b5-4b77-a990-ed052f65e676 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:39.639140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af13d5d-92b5-4b77-a990-ed052f65e676 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:39.640558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:39.645730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea1ac4-7c08-4b00-b80f-719cf2197fd9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:39.646161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea1ac4-7c08-4b00-b80f-719cf2197fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:39.646283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea1ac4-7c08-4b00-b80f-719cf2197fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:39.722377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4cea1ac4-7c08-4b00-b80f-719cf2197fd9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:39.723588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4cea1ac4-7c08-4b00-b80f-719cf2197fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:39.725686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:39.820041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:39.971862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af13d5d-92b5-4b77-a990-ed052f65e676 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:39.973423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3af13d5d-92b5-4b77-a990-ed052f65e676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:40.125002 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af13d5d-92b5-4b77-a990-ed052f65e676 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:40.150892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:40.235985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9222e52-f937-4f73-9951-766e1c00f534 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:40.236818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9222e52-f937-4f73-9951-766e1c00f534 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:40.237101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9222e52-f937-4f73-9951-766e1c00f534 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:40.237389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9222e52-f937-4f73-9951-766e1c00f534 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:40.238992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f9222e52-f937-4f73-9951-766e1c00f534 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:40.327377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9222e52-f937-4f73-9951-766e1c00f534 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:40.328983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9222e52-f937-4f73-9951-766e1c00f534 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:40.346584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 732/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:42.278440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be777-a651-4a18-9bbe-602557c2d130 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:42.278973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be777-a651-4a18-9bbe-602557c2d130 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:42.278973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be777-a651-4a18-9bbe-602557c2d130 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:42.280322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:42.285609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd4812-a639-4c27-9fe3-bedafce8be7c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:42.286027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd4812-a639-4c27-9fe3-bedafce8be7c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:42.286170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd4812-a639-4c27-9fe3-bedafce8be7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:42.325048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:42.349682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-33cd4812-a639-4c27-9fe3-bedafce8be7c None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:42.350867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-33cd4812-a639-4c27-9fe3-bedafce8be7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:42.445199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:42.484368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be777-a651-4a18-9bbe-602557c2d130 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:42.485901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-235be777-a651-4a18-9bbe-602557c2d130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:42.587145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be777-a651-4a18-9bbe-602557c2d130 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:42.620058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:42.661762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2c4d4-ba35-4ef0-be97-c3f1b8dd6945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:42.662417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2c4d4-ba35-4ef0-be97-c3f1b8dd6945 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:42.662708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2c4d4-ba35-4ef0-be97-c3f1b8dd6945 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:42.662990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2c4d4-ba35-4ef0-be97-c3f1b8dd6945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:42.665561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7db2c4d4-ba35-4ef0-be97-c3f1b8dd6945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:42.773853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2c4d4-ba35-4ef0-be97-c3f1b8dd6945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:42.775336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2c4d4-ba35-4ef0-be97-c3f1b8dd6945 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:42.793210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 710/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:44.989669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4830-3529-4613-a5b9-2a7b62038c9f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:44.990277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4830-3529-4613-a5b9-2a7b62038c9f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:44.990277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4830-3529-4613-a5b9-2a7b62038c9f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:44.991579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:44.997363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc3e0f-dd39-4071-86f7-99ab2389261e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:44.997808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc3e0f-dd39-4071-86f7-99ab2389261e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:44.997909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc3e0f-dd39-4071-86f7-99ab2389261e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:45.062765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d8fc3e0f-dd39-4071-86f7-99ab2389261e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:45.064004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d8fc3e0f-dd39-4071-86f7-99ab2389261e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:45.076370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:45.160138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:45.199300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a4830-3529-4613-a5b9-2a7b62038c9f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:45.200867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-392a4830-3529-4613-a5b9-2a7b62038c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:45.302474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a4830-3529-4613-a5b9-2a7b62038c9f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:45.314434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:45.374865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d61f8-f2bd-46ab-8aed-9912a1ed0f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:45.375539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d61f8-f2bd-46ab-8aed-9912a1ed0f28 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:45.375830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d61f8-f2bd-46ab-8aed-9912a1ed0f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:45.376114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d61f8-f2bd-46ab-8aed-9912a1ed0f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:45.377795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-598d61f8-f2bd-46ab-8aed-9912a1ed0f28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:45.484749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d61f8-f2bd-46ab-8aed-9912a1ed0f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:45.486428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d61f8-f2bd-46ab-8aed-9912a1ed0f28 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:45.504061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 735/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:47.894463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0164bc-bcc0-42a0-b57e-7b1cdc49cdd5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:47.895034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0164bc-bcc0-42a0-b57e-7b1cdc49cdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:47.895034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0164bc-bcc0-42a0-b57e-7b1cdc49cdd5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:47.896445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:47.902437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10dea7-10e4-43e4-a7d0-477f8e4c3c61 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:47.902888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10dea7-10e4-43e4-a7d0-477f8e4c3c61 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:47.903005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10dea7-10e4-43e4-a7d0-477f8e4c3c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:47.948877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:47.967695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ac10dea7-10e4-43e4-a7d0-477f8e4c3c61 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:47.968921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ac10dea7-10e4-43e4-a7d0-477f8e4c3c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:48.063550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:48.102575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0164bc-bcc0-42a0-b57e-7b1cdc49cdd5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:48.104247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0d0164bc-bcc0-42a0-b57e-7b1cdc49cdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:48.207954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0164bc-bcc0-42a0-b57e-7b1cdc49cdd5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:48.246446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:48.280151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd032f3-2407-4c26-b3d3-1e9a7f8b5f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:48.280791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd032f3-2407-4c26-b3d3-1e9a7f8b5f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:48.281070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd032f3-2407-4c26-b3d3-1e9a7f8b5f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:48.281342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd032f3-2407-4c26-b3d3-1e9a7f8b5f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:48.282954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3dd032f3-2407-4c26-b3d3-1e9a7f8b5f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:48.388186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd032f3-2407-4c26-b3d3-1e9a7f8b5f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:48.389622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd032f3-2407-4c26-b3d3-1e9a7f8b5f93 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:48.406973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 713/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:50.563879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61e590-e13e-4615-8a74-fbe826e2b7f3 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:50.564420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61e590-e13e-4615-8a74-fbe826e2b7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:50.564420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61e590-e13e-4615-8a74-fbe826e2b7f3 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:50.565798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:50.571093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad879e-8f8b-4432-9e2c-080911505f7b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:50.571537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad879e-8f8b-4432-9e2c-080911505f7b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:50.571715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad879e-8f8b-4432-9e2c-080911505f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:50.636717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-05ad879e-8f8b-4432-9e2c-080911505f7b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:50.637915 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-05ad879e-8f8b-4432-9e2c-080911505f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:50.657576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:50.735011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:50.773849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61e590-e13e-4615-8a74-fbe826e2b7f3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:50.775397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4e61e590-e13e-4615-8a74-fbe826e2b7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:50.877279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61e590-e13e-4615-8a74-fbe826e2b7f3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:50.896327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:50.949019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82d882-5740-41cf-a4c3-a269e2833269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:50.949663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82d882-5740-41cf-a4c3-a269e2833269 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:50.949952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82d882-5740-41cf-a4c3-a269e2833269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:50.950235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82d882-5740-41cf-a4c3-a269e2833269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:50.951901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6c82d882-5740-41cf-a4c3-a269e2833269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:51.059271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82d882-5740-41cf-a4c3-a269e2833269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:51.060721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82d882-5740-41cf-a4c3-a269e2833269 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:51.078471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 738/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:53.160584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd655aa3-e865-406b-9833-9236eaaad153 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:53.161151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd655aa3-e865-406b-9833-9236eaaad153 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:53.161151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd655aa3-e865-406b-9833-9236eaaad153 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:53.162545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:53.167708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3d53f-24c3-4768-8678-a41cc995aff5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:53.168133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3d53f-24c3-4768-8678-a41cc995aff5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:53.168271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3d53f-24c3-4768-8678-a41cc995aff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:53.229334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:53.232363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1ae3d53f-24c3-4768-8678-a41cc995aff5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:53.233886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1ae3d53f-24c3-4768-8678-a41cc995aff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:53.328013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:53.366321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd655aa3-e865-406b-9833-9236eaaad153 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:53.367935 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd655aa3-e865-406b-9833-9236eaaad153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:53.469852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd655aa3-e865-406b-9833-9236eaaad153 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:53.517911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:53.541795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675acff-6749-43c3-93ea-632b62a745e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:53.542437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675acff-6749-43c3-93ea-632b62a745e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:53.542721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675acff-6749-43c3-93ea-632b62a745e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:53.542996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675acff-6749-43c3-93ea-632b62a745e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:53.544689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1675acff-6749-43c3-93ea-632b62a745e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:53.651592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675acff-6749-43c3-93ea-632b62a745e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:53.653087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675acff-6749-43c3-93ea-632b62a745e5 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:53.674396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 716/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:55.776289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddef11-4b77-4378-a22b-cbc2d223a1d2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:55.776801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddef11-4b77-4378-a22b-cbc2d223a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:55.776801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddef11-4b77-4378-a22b-cbc2d223a1d2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:55.778214 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:07:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:55.783540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306468b-61ce-4a29-8068-382f28529a90 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:55.783958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306468b-61ce-4a29-8068-382f28529a90 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:55.784108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306468b-61ce-4a29-8068-382f28529a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:55.847903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1306468b-61ce-4a29-8068-382f28529a90 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:55.849114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1306468b-61ce-4a29-8068-382f28529a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:55.878185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:55.946307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:55.985282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddef11-4b77-4378-a22b-cbc2d223a1d2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:55.987089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2fddef11-4b77-4378-a22b-cbc2d223a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:56.087463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddef11-4b77-4378-a22b-cbc2d223a1d2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:56.115020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:56.158976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6645-abfb-4b76-b8aa-1687c15a5728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:56.159665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6645-abfb-4b76-b8aa-1687c15a5728 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:56.159946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6645-abfb-4b76-b8aa-1687c15a5728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:56.160219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6645-abfb-4b76-b8aa-1687c15a5728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:56.161828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3a6d6645-abfb-4b76-b8aa-1687c15a5728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:56.273809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d6645-abfb-4b76-b8aa-1687c15a5728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:56.275273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d6645-abfb-4b76-b8aa-1687c15a5728 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:56.293432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 741/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:07:58.448534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9107-0dd4-4cee-878d-f8f5e4319f0b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:58.449104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9107-0dd4-4cee-878d-f8f5e4319f0b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:58.449104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9107-0dd4-4cee-878d-f8f5e4319f0b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:58.450463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:07:58.455883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726cd72-78ce-4a7d-b31b-6e5d7edd205f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:58.456311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726cd72-78ce-4a7d-b31b-6e5d7edd205f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:58.456446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726cd72-78ce-4a7d-b31b-6e5d7edd205f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:58.520113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7726cd72-78ce-4a7d-b31b-6e5d7edd205f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:07:58.521328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7726cd72-78ce-4a7d-b31b-6e5d7edd205f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:58.529418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:58.617122 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:07:58.655189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c9107-0dd4-4cee-878d-f8f5e4319f0b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:07:58.656753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b0c9107-0dd4-4cee-878d-f8f5e4319f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:58.759230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c9107-0dd4-4cee-878d-f8f5e4319f0b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:07:58.818285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:07:58.838226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8f927-6143-40b5-9c9c-dce8ae7a910f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:07:58.838879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8f927-6143-40b5-9c9c-dce8ae7a910f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:07:58.839167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8f927-6143-40b5-9c9c-dce8ae7a910f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:07:58.839482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8f927-6143-40b5-9c9c-dce8ae7a910f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:07:58.841119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4fd8f927-6143-40b5-9c9c-dce8ae7a910f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:07:58.948450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8f927-6143-40b5-9c9c-dce8ae7a910f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:07:58.949896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8f927-6143-40b5-9c9c-dce8ae7a910f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:07:58.967367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 719/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:01.064513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e32a0-fbe3-4778-866a-d68110f74f17 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:01.065061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e32a0-fbe3-4778-866a-d68110f74f17 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:01.065061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e32a0-fbe3-4778-866a-d68110f74f17 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:01.066456 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:01.071812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d227c-5f59-40ec-b18d-622d32b01fe1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:01.072279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d227c-5f59-40ec-b18d-622d32b01fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:01.072408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d227c-5f59-40ec-b18d-622d32b01fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:01.141113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-022d227c-5f59-40ec-b18d-622d32b01fe1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:01.142355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-022d227c-5f59-40ec-b18d-622d32b01fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:01.182592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:01.240623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:01.277962 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e32a0-fbe3-4778-866a-d68110f74f17 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:01.279548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-913e32a0-fbe3-4778-866a-d68110f74f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:01.380627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e32a0-fbe3-4778-866a-d68110f74f17 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:01.417030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:01.451956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860fa73-b543-44a5-9ae3-054298b2e22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:01.452606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860fa73-b543-44a5-9ae3-054298b2e22f service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:01.452893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860fa73-b543-44a5-9ae3-054298b2e22f service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:01.453173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860fa73-b543-44a5-9ae3-054298b2e22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:01.454795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9860fa73-b543-44a5-9ae3-054298b2e22f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:01.570267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860fa73-b543-44a5-9ae3-054298b2e22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:01.571738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860fa73-b543-44a5-9ae3-054298b2e22f service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:01.591622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 744/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:03.635637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc862c-d0f4-471f-8709-d83c8af818ab None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:03.636181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc862c-d0f4-471f-8709-d83c8af818ab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:03.636181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc862c-d0f4-471f-8709-d83c8af818ab None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:03.637570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:08:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:03.642920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e168a-1a5a-4b29-a2a1-b776313acaad None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:03.643340 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e168a-1a5a-4b29-a2a1-b776313acaad None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:03.643522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e168a-1a5a-4b29-a2a1-b776313acaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:03.707143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-063e168a-1a5a-4b29-a2a1-b776313acaad None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:03.708395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-063e168a-1a5a-4b29-a2a1-b776313acaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:03.729485 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:03.808321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:03.845236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc862c-d0f4-471f-8709-d83c8af818ab None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:03.846781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-23fc862c-d0f4-471f-8709-d83c8af818ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:03.948345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc862c-d0f4-471f-8709-d83c8af818ab None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:04.013921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:04.021178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9400e36-de8c-4381-ae84-d42e0ab976f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:04.021825 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9400e36-de8c-4381-ae84-d42e0ab976f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:04.022109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9400e36-de8c-4381-ae84-d42e0ab976f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:04.022400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9400e36-de8c-4381-ae84-d42e0ab976f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:04.024062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c9400e36-de8c-4381-ae84-d42e0ab976f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:04.131686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9400e36-de8c-4381-ae84-d42e0ab976f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:04.133139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9400e36-de8c-4381-ae84-d42e0ab976f8 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:04.151563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 722/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:08:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:06.201289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312659-21a2-42ab-8c4f-900ee7a604ae None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:06.201865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312659-21a2-42ab-8c4f-900ee7a604ae None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:06.201865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312659-21a2-42ab-8c4f-900ee7a604ae None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:06.203243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:06.208934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437449f6-7059-44ee-908a-03e414e054b5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:06.209416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437449f6-7059-44ee-908a-03e414e054b5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:06.209679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437449f6-7059-44ee-908a-03e414e054b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:06.274250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-437449f6-7059-44ee-908a-03e414e054b5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:06.276081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-437449f6-7059-44ee-908a-03e414e054b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:06.323563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:06.371642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:08:06.408959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a312659-21a2-42ab-8c4f-900ee7a604ae None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:06.410518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2a312659-21a2-42ab-8c4f-900ee7a604ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:06.511616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a312659-21a2-42ab-8c4f-900ee7a604ae None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:06.557961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:06.584212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc7142-1c64-41e9-b95f-031ac14f6cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:06.584855 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc7142-1c64-41e9-b95f-031ac14f6cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:06.585139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc7142-1c64-41e9-b95f-031ac14f6cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:06.585412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc7142-1c64-41e9-b95f-031ac14f6cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:06.587021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b4fc7142-1c64-41e9-b95f-031ac14f6cc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:06.696658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc7142-1c64-41e9-b95f-031ac14f6cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:06.698600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc7142-1c64-41e9-b95f-031ac14f6cc5 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:06.717461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 747/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:16.369778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=22 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:16.715145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=66 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:26.529622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958783e7-3fd5-4571-be1c-2756f5a6f3fc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:26.530154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958783e7-3fd5-4571-be1c-2756f5a6f3fc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:26.530154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958783e7-3fd5-4571-be1c-2756f5a6f3fc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:26.531610 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 724/1471] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:26.540247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58156de-848a-4487-85a8-a73d5762e0bc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:26.540688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58156de-848a-4487-85a8-a73d5762e0bc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:26.540874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58156de-848a-4487-85a8-a73d5762e0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:26.607494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d58156de-848a-4487-85a8-a73d5762e0bc None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:26.608698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d58156de-848a-4487-85a8-a73d5762e0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:26.702836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 748/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:26.785448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd672dc-05aa-4602-a40d-72e3066fdd93 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:26.785744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd672dc-05aa-4602-a40d-72e3066fdd93 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:26.786026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd672dc-05aa-4602-a40d-72e3066fdd93 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:26.787452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 725/1473] 10.222.0.20 () {54 vars in 966 bytes} [Fri Jul 22 11:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:26.792129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c820a60-934e-4ef9-9954-f6afea657d49 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:26.792421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c820a60-934e-4ef9-9954-f6afea657d49 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:26.792700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c820a60-934e-4ef9-9954-f6afea657d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:26.874858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3c820a60-934e-4ef9-9954-f6afea657d49 None None] MFA Rules not processed for user `92fc25f9ad7c49c48a909d2d92ea0a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:26.876116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3c820a60-934e-4ef9-9954-f6afea657d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:26.897136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:26.952758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 749/1474] 10.222.0.20 () {58 vars in 1063 bytes} [Fri Jul 22 11:08:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:26.960608 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd672dc-05aa-4602-a40d-72e3066fdd93 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:26.962114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1cd672dc-05aa-4602-a40d-72e3066fdd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:27.063337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd672dc-05aa-4602-a40d-72e3066fdd93 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:27.134284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995e5a9-5b89-4e98-aada-71c6d837393b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92fc25f9ad7c49c48a909d2d92ea0a66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:27.135483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995e5a9-5b89-4e98-aada-71c6d837393b service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:27.135766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995e5a9-5b89-4e98-aada-71c6d837393b service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:27.136393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995e5a9-5b89-4e98-aada-71c6d837393b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:27.138026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6995e5a9-5b89-4e98-aada-71c6d837393b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:27.190133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:27.245510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995e5a9-5b89-4e98-aada-71c6d837393b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:27.246948 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995e5a9-5b89-4e98-aada-71c6d837393b service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:27.265496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 726/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:29.381522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903b114-8dad-47d1-8a4b-8ce2cdb85841 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:29.382072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903b114-8dad-47d1-8a4b-8ce2cdb85841 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:29.382072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903b114-8dad-47d1-8a4b-8ce2cdb85841 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:29.383502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 750/1476] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:29.389242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a891efe-f96f-4ff5-81b7-0029969af802 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:29.389645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a891efe-f96f-4ff5-81b7-0029969af802 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:29.389806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a891efe-f96f-4ff5-81b7-0029969af802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:29.453870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9a891efe-f96f-4ff5-81b7-0029969af802 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:29.455091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9a891efe-f96f-4ff5-81b7-0029969af802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:29.550071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 727/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:29.573910 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903b114-8dad-47d1-8a4b-8ce2cdb85841 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:29.575470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5903b114-8dad-47d1-8a4b-8ce2cdb85841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:29.699645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903b114-8dad-47d1-8a4b-8ce2cdb85841 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:29.746559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:29.791400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddca54-65ad-4919-a45c-10ad34bac53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:29.792054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddca54-65ad-4919-a45c-10ad34bac53f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:29.792335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddca54-65ad-4919-a45c-10ad34bac53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:29.792610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddca54-65ad-4919-a45c-10ad34bac53f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:29.793338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddca54-65ad-4919-a45c-10ad34bac53f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:29.795035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-ddddca54-65ad-4919-a45c-10ad34bac53f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:08:29.795035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 22 11:08:29.795035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: Jul 22 11:08:29.796197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddca54-65ad-4919-a45c-10ad34bac53f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:29.814440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 751/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Jul 22 11:08:29 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:08:31.862515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4decf68-97fd-4f08-97c3-dd88fff710ab None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:31.863052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4decf68-97fd-4f08-97c3-dd88fff710ab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:31.863172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4decf68-97fd-4f08-97c3-dd88fff710ab None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:31.864506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 728/1479] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:31.870244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22d3d6-8da4-4a8b-a256-a2155889f9dd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:31.870686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22d3d6-8da4-4a8b-a256-a2155889f9dd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:31.870794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22d3d6-8da4-4a8b-a256-a2155889f9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:31.935569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fc22d3d6-8da4-4a8b-a256-a2155889f9dd None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:31.936783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fc22d3d6-8da4-4a8b-a256-a2155889f9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:32.031178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 752/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:32.075102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4decf68-97fd-4f08-97c3-dd88fff710ab None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:32.076653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f4decf68-97fd-4f08-97c3-dd88fff710ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:32.087664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:32.221982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4decf68-97fd-4f08-97c3-dd88fff710ab None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:32.301339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:32.333558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de544b-084a-4c34-bf79-5a6b2905db35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:32.334203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de544b-084a-4c34-bf79-5a6b2905db35 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:32.334495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de544b-084a-4c34-bf79-5a6b2905db35 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:32.334771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de544b-084a-4c34-bf79-5a6b2905db35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:32.336455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-57de544b-084a-4c34-bf79-5a6b2905db35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:32.425264 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de544b-084a-4c34-bf79-5a6b2905db35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:32.426864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de544b-084a-4c34-bf79-5a6b2905db35 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:32.444528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 729/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:34.644397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fb625-cd13-4d44-a89c-76b87d4043c9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:34.644906 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fb625-cd13-4d44-a89c-76b87d4043c9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:34.645018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fb625-cd13-4d44-a89c-76b87d4043c9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:34.646337 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 753/1482] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:34.651617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46a3f6-36f0-49c5-b505-5a55c98e3cc1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:34.652056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46a3f6-36f0-49c5-b505-5a55c98e3cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:34.652221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46a3f6-36f0-49c5-b505-5a55c98e3cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:34.715586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-aa46a3f6-36f0-49c5-b505-5a55c98e3cc1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:34.716815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-aa46a3f6-36f0-49c5-b505-5a55c98e3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:34.762353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:34.812842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 730/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:34.850375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fb625-cd13-4d44-a89c-76b87d4043c9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:34.851881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8a7fb625-cd13-4d44-a89c-76b87d4043c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:34.876374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:34.956168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fb625-cd13-4d44-a89c-76b87d4043c9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:35.028850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774ba33-3c72-432b-8deb-b4b9ae6fb99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:35.029508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774ba33-3c72-432b-8deb-b4b9ae6fb99e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:35.029798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774ba33-3c72-432b-8deb-b4b9ae6fb99e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:35.030081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774ba33-3c72-432b-8deb-b4b9ae6fb99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:35.031754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c774ba33-3c72-432b-8deb-b4b9ae6fb99e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:35.105148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:35.140631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774ba33-3c72-432b-8deb-b4b9ae6fb99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:35.142070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774ba33-3c72-432b-8deb-b4b9ae6fb99e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:35.161851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 754/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:37.233569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf1d55-51ac-449e-9795-550dc08dd473 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:37.234143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf1d55-51ac-449e-9795-550dc08dd473 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:37.234143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf1d55-51ac-449e-9795-550dc08dd473 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:37.235553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 731/1485] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:37.240949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04750581-24c2-43fd-974b-ed53319789ec None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:37.241382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04750581-24c2-43fd-974b-ed53319789ec None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:37.241490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04750581-24c2-43fd-974b-ed53319789ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:37.304732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-04750581-24c2-43fd-974b-ed53319789ec None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:37.305930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-04750581-24c2-43fd-974b-ed53319789ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:37.404155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 755/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:37.442798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf1d55-51ac-449e-9795-550dc08dd473 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:37.444424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0ebf1d55-51ac-449e-9795-550dc08dd473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:37.546896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf1d55-51ac-449e-9795-550dc08dd473 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:37.624739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:37.626790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f31bd4-e564-4050-a607-d6c81d5df71e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:37.627499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f31bd4-e564-4050-a607-d6c81d5df71e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:37.627984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f31bd4-e564-4050-a607-d6c81d5df71e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:37.628426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f31bd4-e564-4050-a607-d6c81d5df71e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:37.630235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b6f31bd4-e564-4050-a607-d6c81d5df71e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:37.737579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f31bd4-e564-4050-a607-d6c81d5df71e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:37.739026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f31bd4-e564-4050-a607-d6c81d5df71e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:37.757132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 732/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:39.952347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31739f3-9bcc-4882-b8c9-ed8688ef7660 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:39.952914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31739f3-9bcc-4882-b8c9-ed8688ef7660 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:39.952914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31739f3-9bcc-4882-b8c9-ed8688ef7660 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:39.954313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 756/1488] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:39.959421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9129ab-527a-4c71-9709-361cb8571bdb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:39.959852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9129ab-527a-4c71-9709-361cb8571bdb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:39.959983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9129ab-527a-4c71-9709-361cb8571bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:40.024104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3d9129ab-527a-4c71-9709-361cb8571bdb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:40.025858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3d9129ab-527a-4c71-9709-361cb8571bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:40.077933 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:40.121833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 733/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:40.163165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31739f3-9bcc-4882-b8c9-ed8688ef7660 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:40.164764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a31739f3-9bcc-4882-b8c9-ed8688ef7660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:40.200985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:40.268654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31739f3-9bcc-4882-b8c9-ed8688ef7660 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:40.339929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e60bd-23cd-4611-b6a9-f9c84d240558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:40.340576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e60bd-23cd-4611-b6a9-f9c84d240558 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:40.340860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e60bd-23cd-4611-b6a9-f9c84d240558 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:40.341136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e60bd-23cd-4611-b6a9-f9c84d240558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:40.342752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-523e60bd-23cd-4611-b6a9-f9c84d240558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:40.420614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:40.449393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e60bd-23cd-4611-b6a9-f9c84d240558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:40.450819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e60bd-23cd-4611-b6a9-f9c84d240558 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:40.469129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 757/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:08:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:42.716469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666554e-a6e6-465d-a890-aca3b48c6393 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:42.717028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666554e-a6e6-465d-a890-aca3b48c6393 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:42.717028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666554e-a6e6-465d-a890-aca3b48c6393 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:42.718415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 734/1491] 10.222.0.20 () {54 vars in 926 bytes} [Fri Jul 22 11:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:08:42.723587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08629e3e-63a8-4c84-b56f-7351505781d5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:42.723971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08629e3e-63a8-4c84-b56f-7351505781d5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:42.724157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08629e3e-63a8-4c84-b56f-7351505781d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:42.789506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-08629e3e-63a8-4c84-b56f-7351505781d5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:08:42.791378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-08629e3e-63a8-4c84-b56f-7351505781d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:42.885085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 758/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Fri Jul 22 11:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:08:42.924795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666554e-a6e6-465d-a890-aca3b48c6393 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:08:42.926315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4666554e-a6e6-465d-a890-aca3b48c6393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:43.027977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666554e-a6e6-465d-a890-aca3b48c6393 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:08:43.098270 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352892e2-8c81-426b-bcfd-eb898e97e19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:08:43.098911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352892e2-8c81-426b-bcfd-eb898e97e19d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:08:43.099194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352892e2-8c81-426b-bcfd-eb898e97e19d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:08:43.099499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352892e2-8c81-426b-bcfd-eb898e97e19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:08:43.101107 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-352892e2-8c81-426b-bcfd-eb898e97e19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:08:43.110464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:43.209820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352892e2-8c81-426b-bcfd-eb898e97e19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:08:43.211185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352892e2-8c81-426b-bcfd-eb898e97e19d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:08:43.229375 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 735/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:08:52.883290 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=82 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:08:53.226967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=50 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:11.065908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea0480-e090-45bb-9b8c-7cad4a3731da None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:11.066480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea0480-e090-45bb-9b8c-7cad4a3731da None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:11.066480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea0480-e090-45bb-9b8c-7cad4a3731da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:11.138738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-16ea0480-e090-45bb-9b8c-7cad4a3731da None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:10:11.139966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-16ea0480-e090-45bb-9b8c-7cad4a3731da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:11.236035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 759/1494] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:10:11.245132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d0db5-84fa-417d-b2a4-4a860ace7d41 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:11.245615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d0db5-84fa-417d-b2a4-4a860ace7d41 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:11.245719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d0db5-84fa-417d-b2a4-4a860ace7d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:11.320410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-212d0db5-84fa-417d-b2a4-4a860ace7d41 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:10:11.321877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-212d0db5-84fa-417d-b2a4-4a860ace7d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:11.428463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 736/1495] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:10:11.433179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea0480-e090-45bb-9b8c-7cad4a3731da None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:11.434393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-16ea0480-e090-45bb-9b8c-7cad4a3731da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:11.445988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea0480-e090-45bb-9b8c-7cad4a3731da None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:11.511733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:11.546844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:11.547549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:11.547837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:11.548115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:11.548866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:10:11.549197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:11.550207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:10:11.550207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 22 11:10:11.550207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: Jul 22 11:10:11.551127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:11.562584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 760/1496] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:10:11.571194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d0db5-84fa-417d-b2a4-4a860ace7d41 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:11.580475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d0db5-84fa-417d-b2a4-4a860ace7d41 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:11.641955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:11.676205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:11.676869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:11.677156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:11.677435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:11.678121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:11.680023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:10:11.680023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 22 11:10:11.680023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: Jul 22 11:10:11.681234 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:11.734807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 737/1497] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:10:11 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:10:11.739346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:11.740908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:11.774121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49810eb8-a62b-48e4-a1b4-aeee08d0ba3f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:11.866206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:11.866796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:11.867082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:11.867395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:11.868084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:11.869092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:10:11.869092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 22 11:10:11.869092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: Jul 22 11:10:11.870293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:11.911416 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 761/1498] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:10:11 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:10:11.915467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:11.916973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:11.977408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5a814-c825-4927-8eef-40f6b6fa6e16 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:12.067623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:12.080367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:12.081094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:12.081383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:12.081655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:12.082377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:12.083446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:10:12.083446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 22 11:10:12.083446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: Jul 22 11:10:12.084341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:12.096619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 738/1499] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:10:12.100607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:12.102156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:12.113806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7cfe9-73b9-4f17-be6a-09a12e5d6466 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:12.139948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:12.209569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd2417-ba9f-402a-b372-6e7b59bc22d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:12.210265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd2417-ba9f-402a-b372-6e7b59bc22d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:12.210552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd2417-ba9f-402a-b372-6e7b59bc22d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:12.210828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd2417-ba9f-402a-b372-6e7b59bc22d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:12.211596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd2417-ba9f-402a-b372-6e7b59bc22d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:12.212608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-6dbd2417-ba9f-402a-b372-6e7b59bc22d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:10:12.212608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 22 11:10:12.212608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: Jul 22 11:10:12.213498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd2417-ba9f-402a-b372-6e7b59bc22d7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:12.225785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 762/1500] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:10:12.232407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:12.233995 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:12.268426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ab44b-d9fb-4322-9210-99d002b5e0d4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:12.371976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343001ac-c8c3-46fc-b26c-ab4f664b3518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:12.372868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343001ac-c8c3-46fc-b26c-ab4f664b3518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:12.373155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343001ac-c8c3-46fc-b26c-ab4f664b3518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:12.373434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343001ac-c8c3-46fc-b26c-ab4f664b3518 admin admin] PATH_INFO: `/v3/projects/ea8909d0cba544a1bcb9573412856a8f/users/20b52e4b8f16448c90a7a96fc0275058/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:12.395405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343001ac-c8c3-46fc-b26c-ab4f664b3518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8909d0cba544a1bcb9573412856a8f, user_id=20b52e4b8f16448c90a7a96fc0275058, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:12.396393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-343001ac-c8c3-46fc-b26c-ab4f664b3518 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:10:12.396393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 22 11:10:12.396393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: Jul 22 11:10:12.398366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343001ac-c8c3-46fc-b26c-ab4f664b3518 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:12.540729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 739/1501] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:10:12 2022] PUT /identity/v3/projects/ea8909d0cba544a1bcb9573412856a8f/users/20b52e4b8f16448c90a7a96fc0275058/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:10:12.552858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad635842-f3c1-4d2d-9cb1-c5379828aaf5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:12.553222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad635842-f3c1-4d2d-9cb1-c5379828aaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:12.554140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad635842-f3c1-4d2d-9cb1-c5379828aaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:12.611933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ad635842-f3c1-4d2d-9cb1-c5379828aaf5 None None] MFA Rules not processed for user `20b52e4b8f16448c90a7a96fc0275058`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:10:12.614098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ad635842-f3c1-4d2d-9cb1-c5379828aaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:12.705778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 763/1502] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:10:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:10:12.719115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d99542-14f9-428a-82f0-6c35a79742a7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:12.719440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d99542-14f9-428a-82f0-6c35a79742a7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:12.719723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d99542-14f9-428a-82f0-6c35a79742a7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:12.721102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 740/1503] 10.222.0.20 () {54 vars in 964 bytes} [Fri Jul 22 11:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:10:12.726602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f4721-c436-41a9-9640-bfd9a23b8564 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:12.726923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f4721-c436-41a9-9640-bfd9a23b8564 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:12.727203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f4721-c436-41a9-9640-bfd9a23b8564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:12.756564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:12.801452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2a0f4721-c436-41a9-9640-bfd9a23b8564 None None] MFA Rules not processed for user `5b04cb4117974f8eb9bbf3a41c48a11a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:10:12.802648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2a0f4721-c436-41a9-9640-bfd9a23b8564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:12.877647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 764/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Fri Jul 22 11:10:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:10:12.885218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d99542-14f9-428a-82f0-6c35a79742a7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:12.886749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e3d99542-14f9-428a-82f0-6c35a79742a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:12.935520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:13.004568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d99542-14f9-428a-82f0-6c35a79742a7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:13.092561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:13.093200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:13.093480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:13.093748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:13.095458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:13.189844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:13.191287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:13.195248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:13.210765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 741/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:10:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:10:13.252044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdde5c-bfd2-419b-9ef1-58901e6ffaa2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:13.252455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdde5c-bfd2-419b-9ef1-58901e6ffaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:13.252614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdde5c-bfd2-419b-9ef1-58901e6ffaa2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:13.253929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 765/1506] 10.222.0.20 () {52 vars in 829 bytes} [Fri Jul 22 11:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:10:13.258093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:13.259282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:13.354965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35cdb6-9457-417b-b375-5ee9135cb94c service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:13.453402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:13.454271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:13.454557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:13.454834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] PATH_INFO: `/v3/users/20b52e4b8f16448c90a7a96fc0275058` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:13.468077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b52e4b8f16448c90a7a96fc0275058)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:13.469116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:10:13.469116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 22 11:10:13.469116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: Jul 22 11:10:13.470320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:13.479868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:13.532131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b52e4b8f16448c90a7a96fc0275058'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:13.541245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b52e4b8f16448c90a7a96fc0275058'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:13.554125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b52e4b8f16448c90a7a96fc0275058'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:13.572438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b52e4b8f16448c90a7a96fc0275058'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:13.572892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b52e4b8f16448c90a7a96fc0275058'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:13.578465 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 742/1507] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:10:13 2022] DELETE /identity/v3/users/20b52e4b8f16448c90a7a96fc0275058 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:10:13.582920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe5abe-e20f-4952-ac44-88c198e8d9ae None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:13.583220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe5abe-e20f-4952-ac44-88c198e8d9ae None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:13.583543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe5abe-e20f-4952-ac44-88c198e8d9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:13.644987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6bfe5abe-e20f-4952-ac44-88c198e8d9ae None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:10:13.646208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6bfe5abe-e20f-4952-ac44-88c198e8d9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:13.721297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:13.735431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 766/1508] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:10:13.745715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:13.746971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:13.848515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb77f7-de5b-4926-a0cf-ae38f452837d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:13.903541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:13.924496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc70f4-1da1-4de8-b9de-f7f859fbd270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:13.925148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc70f4-1da1-4de8-b9de-f7f859fbd270 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:13.925439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc70f4-1da1-4de8-b9de-f7f859fbd270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:13.925721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc70f4-1da1-4de8-b9de-f7f859fbd270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:13.927480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-49fc70f4-1da1-4de8-b9de-f7f859fbd270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:14.037403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc70f4-1da1-4de8-b9de-f7f859fbd270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:14.039404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc70f4-1da1-4de8-b9de-f7f859fbd270 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:14.058592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 743/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:10:14.384273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe5abe-e20f-4952-ac44-88c198e8d9ae None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:10:14.385513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6bfe5abe-e20f-4952-ac44-88c198e8d9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:10:14.397762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe5abe-e20f-4952-ac44-88c198e8d9ae None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:10:14.494914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:10:14.495805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:10:14.496095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:10:14.496377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] PATH_INFO: `/v3/projects/ea8909d0cba544a1bcb9573412856a8f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:10:14.506095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8909d0cba544a1bcb9573412856a8f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:10:14.507152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:10:14.507152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 22 11:10:14.507152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: Jul 22 11:10:14.508376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:10:14.565815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8909d0cba544a1bcb9573412856a8f', 'request_id': 'req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:14.571914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8909d0cba544a1bcb9573412856a8f', 'request_id': 'req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:14.584572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8909d0cba544a1bcb9573412856a8f', 'request_id': 'req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:14.613961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] The token cache is being invalidate because project ea8909d0cba544a1bcb9573412856a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:10:14.614255 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:14.614683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3ca9ab3-c24c-48be-9590-4df4c3dd5b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:10:14.616169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 767/1510] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:10:14 2022] DELETE /identity/v3/projects/ea8909d0cba544a1bcb9573412856a8f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:10:24.056239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=62 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:10:24.611653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:11:52.564562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b01ec2-cbac-49ee-89f6-11399c7fd330 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:52.565219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b01ec2-cbac-49ee-89f6-11399c7fd330 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:52.565327 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b01ec2-cbac-49ee-89f6-11399c7fd330 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:52.567192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:11:52.576544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e60a1b-9e57-4d37-9f4b-b0860e1e285b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:52.577357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e60a1b-9e57-4d37-9f4b-b0860e1e285b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:52.577692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e60a1b-9e57-4d37-9f4b-b0860e1e285b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:52.724712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-42e60a1b-9e57-4d37-9f4b-b0860e1e285b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:11:52.726810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-42e60a1b-9e57-4d37-9f4b-b0860e1e285b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:11:52.909252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:11:53.073585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b01ec2-cbac-49ee-89f6-11399c7fd330 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:11:53.076105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-16b01ec2-cbac-49ee-89f6-11399c7fd330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:11:53.227622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b01ec2-cbac-49ee-89f6-11399c7fd330 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:11:53.233658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803d7d8-1cff-4bac-9937-ac010ba77da2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:11:53.234377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803d7d8-1cff-4bac-9937-ac010ba77da2 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:53.234739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803d7d8-1cff-4bac-9937-ac010ba77da2 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:53.235070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803d7d8-1cff-4bac-9937-ac010ba77da2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:53.245559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803d7d8-1cff-4bac-9937-ac010ba77da2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:11:53.247198 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803d7d8-1cff-4bac-9937-ac010ba77da2 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:11:53.258862 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 745/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:11:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:11:54.137579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31130e4e-0df2-4059-9540-3e59051664d0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:54.138322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31130e4e-0df2-4059-9540-3e59051664d0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:54.138322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31130e4e-0df2-4059-9540-3e59051664d0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:54.139659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:11:54.145184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736f60b-4009-4e04-a99e-eb5565507655 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:54.145702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736f60b-4009-4e04-a99e-eb5565507655 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:54.145848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736f60b-4009-4e04-a99e-eb5565507655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:54.172739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5736f60b-4009-4e04-a99e-eb5565507655 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:11:54.174680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5736f60b-4009-4e04-a99e-eb5565507655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:11:54.188253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:11:54.355557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31130e4e-0df2-4059-9540-3e59051664d0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:11:54.373397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31130e4e-0df2-4059-9540-3e59051664d0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:11:54.379410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60ef2e-3b4d-4b40-b9e0-e86b9960d596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:11:54.380099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60ef2e-3b4d-4b40-b9e0-e86b9960d596 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:54.380394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60ef2e-3b4d-4b40-b9e0-e86b9960d596 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:54.380727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60ef2e-3b4d-4b40-b9e0-e86b9960d596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:54.391346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60ef2e-3b4d-4b40-b9e0-e86b9960d596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:11:54.392878 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60ef2e-3b4d-4b40-b9e0-e86b9960d596 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:11:54.403151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 770/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:11:54.787769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2211a0f-2da7-4668-aec1-3cd92ef01af1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:54.787979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2211a0f-2da7-4668-aec1-3cd92ef01af1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:54.788256 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2211a0f-2da7-4668-aec1-3cd92ef01af1 None None] PATH_INFO: `/v3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:54.789587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 747/1517] 10.222.0.43 () {54 vars in 925 bytes} [Fri Jul 22 11:11:54 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:11:54.796451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da817890-1f81-4e6c-8e25-61ba2c488491 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:54.796773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da817890-1f81-4e6c-8e25-61ba2c488491 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:54.797054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da817890-1f81-4e6c-8e25-61ba2c488491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:54.857896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-da817890-1f81-4e6c-8e25-61ba2c488491 None None] MFA Rules not processed for user `88ecd4c645174d28930a44b93cfee9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:11:54.859213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-da817890-1f81-4e6c-8e25-61ba2c488491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:11:54.878456 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:11:54.934707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 771/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jul 22 11:11:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:11:58.790055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb9e28-2901-441c-bcc2-9cfeb5425ece None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:58.790570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb9e28-2901-441c-bcc2-9cfeb5425ece None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:58.790667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb9e28-2901-441c-bcc2-9cfeb5425ece None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:58.792112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 748/1519] 10.222.0.20 () {54 vars in 961 bytes} [Fri Jul 22 11:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:11:58.797089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e989aba-0ce2-4421-b6b8-47f81ae73378 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:58.797535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e989aba-0ce2-4421-b6b8-47f81ae73378 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:58.797674 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e989aba-0ce2-4421-b6b8-47f81ae73378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:58.816539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5e989aba-0ce2-4421-b6b8-47f81ae73378 None None] MFA Rules not processed for user `88ecd4c645174d28930a44b93cfee9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:11:58.818378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5e989aba-0ce2-4421-b6b8-47f81ae73378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:11:58.827232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 772/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Fri Jul 22 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:11:58.834897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb9e28-2901-441c-bcc2-9cfeb5425ece None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:11:58.849300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb9e28-2901-441c-bcc2-9cfeb5425ece None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:11:58.854244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96339145-0907-4126-9471-90f4bdac1291 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:11:58.854888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339145-0907-4126-9471-90f4bdac1291 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:11:58.855171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339145-0907-4126-9471-90f4bdac1291 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:11:58.855480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339145-0907-4126-9471-90f4bdac1291 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:11:58.869006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96339145-0907-4126-9471-90f4bdac1291 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:11:58.870670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96339145-0907-4126-9471-90f4bdac1291 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:11:58.880643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 749/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:11:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:12:05.664603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb02b-bbb2-454b-8f7a-ac5adc23bdf3 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:05.665161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb02b-bbb2-454b-8f7a-ac5adc23bdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:05.665161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb02b-bbb2-454b-8f7a-ac5adc23bdf3 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:05.666541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:12:05.671303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62356c-e6c9-4ebf-a374-3e60fb2af70d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:05.671734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62356c-e6c9-4ebf-a374-3e60fb2af70d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:05.671871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62356c-e6c9-4ebf-a374-3e60fb2af70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:05.689815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ba62356c-e6c9-4ebf-a374-3e60fb2af70d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:12:05.691009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba62356c-e6c9-4ebf-a374-3e60fb2af70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:12:05.698725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:12:05.852526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb02b-bbb2-454b-8f7a-ac5adc23bdf3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:12:05.867018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb02b-bbb2-454b-8f7a-ac5adc23bdf3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:12:05.872294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076b595-eaa2-4ac7-9027-cf8c97f58b04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:12:05.872925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076b595-eaa2-4ac7-9027-cf8c97f58b04 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:05.873211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076b595-eaa2-4ac7-9027-cf8c97f58b04 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:05.873488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076b595-eaa2-4ac7-9027-cf8c97f58b04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:05.881493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076b595-eaa2-4ac7-9027-cf8c97f58b04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:12:05.883063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076b595-eaa2-4ac7-9027-cf8c97f58b04 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:12:05.892347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 774/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:12:06.730682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e11f-2b69-48f2-bab0-021871f696cf None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:06.731215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e11f-2b69-48f2-bab0-021871f696cf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:06.731360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e11f-2b69-48f2-bab0-021871f696cf None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:06.732871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 22 11:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:12:06.737783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1ca87-aa8a-42da-9e8f-575a6fb7d95d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:06.738216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1ca87-aa8a-42da-9e8f-575a6fb7d95d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:06.739098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1ca87-aa8a-42da-9e8f-575a6fb7d95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:06.762022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3af1ca87-aa8a-42da-9e8f-575a6fb7d95d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:12:06.763244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3af1ca87-aa8a-42da-9e8f-575a6fb7d95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:12:06.771160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 22 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:12:06.925553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e11f-2b69-48f2-bab0-021871f696cf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:12:06.940219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e11f-2b69-48f2-bab0-021871f696cf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:12:06.946257 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665b557-0a7d-4ae3-95d9-8d712f491517 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:12:06.946886 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b557-0a7d-4ae3-95d9-8d712f491517 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:06.947168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b557-0a7d-4ae3-95d9-8d712f491517 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:06.947488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b557-0a7d-4ae3-95d9-8d712f491517 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:06.955499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665b557-0a7d-4ae3-95d9-8d712f491517 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:12:06.957086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665b557-0a7d-4ae3-95d9-8d712f491517 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:12:06.966367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 752/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:12:10.975585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9726cd-01b9-404c-ab98-dc49a8664b55 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:10.976092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9726cd-01b9-404c-ab98-dc49a8664b55 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:10.976092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9726cd-01b9-404c-ab98-dc49a8664b55 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:10.977503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:12:10.982755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f272eeb-49cd-4a6f-ace5-b76b8892761f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:10.983237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f272eeb-49cd-4a6f-ace5-b76b8892761f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:10.983351 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f272eeb-49cd-4a6f-ace5-b76b8892761f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:11.001244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2f272eeb-49cd-4a6f-ace5-b76b8892761f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:12:11.002439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f272eeb-49cd-4a6f-ace5-b76b8892761f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:12:11.010766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:12:11.083155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9726cd-01b9-404c-ab98-dc49a8664b55 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:12:11.084854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bb9726cd-01b9-404c-ab98-dc49a8664b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:12:11.162446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9726cd-01b9-404c-ab98-dc49a8664b55 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:12:11.166026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae95e5-9ac4-4b98-8fe7-332f197bcce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:12:11.166673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae95e5-9ac4-4b98-8fe7-332f197bcce0 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:11.166963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae95e5-9ac4-4b98-8fe7-332f197bcce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:11.167245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae95e5-9ac4-4b98-8fe7-332f197bcce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:11.176098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae95e5-9ac4-4b98-8fe7-332f197bcce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:12:11.177558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae95e5-9ac4-4b98-8fe7-332f197bcce0 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:12:11.186833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 777/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:12:13.190919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a36bd-6655-4ed6-9619-23388e757c32 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:13.191584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a36bd-6655-4ed6-9619-23388e757c32 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:13.191584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a36bd-6655-4ed6-9619-23388e757c32 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:13.192954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:12:13.198278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54a62f-7a16-4670-8879-88940dc1739c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:13.198723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54a62f-7a16-4670-8879-88940dc1739c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:13.198847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54a62f-7a16-4670-8879-88940dc1739c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:13.219061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ad54a62f-7a16-4670-8879-88940dc1739c None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:12:13.220332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ad54a62f-7a16-4670-8879-88940dc1739c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:12:13.229098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:12:13.266873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a36bd-6655-4ed6-9619-23388e757c32 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:12:13.277950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a36bd-6655-4ed6-9619-23388e757c32 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:12:13.281475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b61b1c-1dda-4f52-8a49-e9fa71bdd90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:12:13.282129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b61b1c-1dda-4f52-8a49-e9fa71bdd90f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:13.282413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b61b1c-1dda-4f52-8a49-e9fa71bdd90f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:13.282687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b61b1c-1dda-4f52-8a49-e9fa71bdd90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:13.292769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b61b1c-1dda-4f52-8a49-e9fa71bdd90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:12:13.294229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b61b1c-1dda-4f52-8a49-e9fa71bdd90f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:12:13.304670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 755/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:12:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:12:16.770685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515276ea-e9d4-41d2-865a-f51a418d33a5 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:16.771366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515276ea-e9d4-41d2-865a-f51a418d33a5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:16.771366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515276ea-e9d4-41d2-865a-f51a418d33a5 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:16.772702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:12:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:12:16.778339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f2867-d51d-4ae6-af7d-01a26b03962f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:16.778830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f2867-d51d-4ae6-af7d-01a26b03962f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:16.778982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f2867-d51d-4ae6-af7d-01a26b03962f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:16.786096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:12:16.798902 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ae6f2867-d51d-4ae6-af7d-01a26b03962f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:12:16.800152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ae6f2867-d51d-4ae6-af7d-01a26b03962f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:12:16.808125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:12:17.118973 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515276ea-e9d4-41d2-865a-f51a418d33a5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:12:17.120670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-515276ea-e9d4-41d2-865a-f51a418d33a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:12:17.184521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:12:17.198469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515276ea-e9d4-41d2-865a-f51a418d33a5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:12:17.202143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a4563-a051-4653-98b9-55b2b2c21a35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:12:17.202779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a4563-a051-4653-98b9-55b2b2c21a35 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:17.203069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a4563-a051-4653-98b9-55b2b2c21a35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:17.203393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a4563-a051-4653-98b9-55b2b2c21a35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:17.212913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a4563-a051-4653-98b9-55b2b2c21a35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:12:17.214342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a4563-a051-4653-98b9-55b2b2c21a35 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:12:17.223873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 780/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:12:19.271688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523ad37-9f57-4824-a8fa-8563a66395a1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:19.272295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523ad37-9f57-4824-a8fa-8563a66395a1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:19.272295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523ad37-9f57-4824-a8fa-8563a66395a1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:19.273699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 22 11:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:12:19.278827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1773d2-7976-416f-9104-bc4ed89d47fb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:19.279305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1773d2-7976-416f-9104-bc4ed89d47fb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:19.279437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1773d2-7976-416f-9104-bc4ed89d47fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:19.297004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-da1773d2-7976-416f-9104-bc4ed89d47fb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:12:19.298193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-da1773d2-7976-416f-9104-bc4ed89d47fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:12:19.306033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 22 11:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:12:19.612704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523ad37-9f57-4824-a8fa-8563a66395a1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:12:19.615125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c523ad37-9f57-4824-a8fa-8563a66395a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:12:19.628380 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523ad37-9f57-4824-a8fa-8563a66395a1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:12:19.632563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d240e-e348-4a8f-b716-13795c41aace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:12:19.633210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d240e-e348-4a8f-b716-13795c41aace service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:12:19.633500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d240e-e348-4a8f-b716-13795c41aace service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:12:19.633781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d240e-e348-4a8f-b716-13795c41aace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:12:19.642829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d240e-e348-4a8f-b716-13795c41aace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:12:19.644304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d240e-e348-4a8f-b716-13795c41aace service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:12:19.653711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 758/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:12:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:12:29.292465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=11 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:12:29.649521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:48.419963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad47a32-71c6-4fd3-ab6f-ee4778f65313 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.420620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad47a32-71c6-4fd3-ab6f-ee4778f65313 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.420620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad47a32-71c6-4fd3-ab6f-ee4778f65313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.445502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f8076-4e0b-4b94-9488-292b534c11b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.447382 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f8076-4e0b-4b94-9488-292b534c11b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.447382 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f8076-4e0b-4b94-9488-292b534c11b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.451304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9ad47a32-71c6-4fd3-ab6f-ee4778f65313 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:48.454916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9ad47a32-71c6-4fd3-ab6f-ee4778f65313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:48.488686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 782/1540] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:13:48.489420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4d1f8076-4e0b-4b94-9488-292b534c11b9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:48.490660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4d1f8076-4e0b-4b94-9488-292b534c11b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:48.497940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0fe19-a4cc-4d22-9e03-021a4bb3bfbd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.498243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0fe19-a4cc-4d22-9e03-021a4bb3bfbd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.498655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0fe19-a4cc-4d22-9e03-021a4bb3bfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.504567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 759/1541] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:48.513412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9852d-1f5c-454f-ade9-11abb1ed6c28 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.513860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9852d-1f5c-454f-ade9-11abb1ed6c28 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.514301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9852d-1f5c-454f-ade9-11abb1ed6c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.528539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cfd0fe19-a4cc-4d22-9e03-021a4bb3bfbd None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:48.529765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cfd0fe19-a4cc-4d22-9e03-021a4bb3bfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:48.537390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f0a9852d-1f5c-454f-ade9-11abb1ed6c28 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:48.538606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f0a9852d-1f5c-454f-ade9-11abb1ed6c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:48.552547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 783/1542] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:13:48.554048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 760/1543] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:48.559320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0fe19-a4cc-4d22-9e03-021a4bb3bfbd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:48.568634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9852d-1f5c-454f-ade9-11abb1ed6c28 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:48.575533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0fe19-a4cc-4d22-9e03-021a4bb3bfbd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:48.583901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9852d-1f5c-454f-ade9-11abb1ed6c28 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:48.587111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff558fb-1256-4f0c-abfd-719e4120e4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:48.587824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff558fb-1256-4f0c-abfd-719e4120e4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.588115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff558fb-1256-4f0c-abfd-719e4120e4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.588399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff558fb-1256-4f0c-abfd-719e4120e4a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.589144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff558fb-1256-4f0c-abfd-719e4120e4a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:13:48.589372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:48.589575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff558fb-1256-4f0c-abfd-719e4120e4a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:48.589963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.590259 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.590540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.591067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff558fb-1256-4f0c-abfd-719e4120e4a7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:48.600938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:13:48.601512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:48.602819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:13:48.602819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 22 11:13:48.602819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: Jul 22 11:13:48.604206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:48.619025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 761/1544] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:13:48.622903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 784/1545] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:13:48.631612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:48.631807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff558fb-1256-4f0c-abfd-719e4120e4a7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:48.642866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ec39d-a500-4f89-bc0d-7a5bed9330b7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:48.647402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff558fb-1256-4f0c-abfd-719e4120e4a7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:48.658779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f7fc9-98a3-4053-baaa-6f71d1d64271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:48.664184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01399cf-39d3-4db1-9cce-6a49d5298afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:48.664869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01399cf-39d3-4db1-9cce-6a49d5298afa admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.665159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01399cf-39d3-4db1-9cce-6a49d5298afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.665445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01399cf-39d3-4db1-9cce-6a49d5298afa admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.666156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01399cf-39d3-4db1-9cce-6a49d5298afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:48.667190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-c01399cf-39d3-4db1-9cce-6a49d5298afa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:13:48.667190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 22 11:13:48.667190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: Jul 22 11:13:48.668256 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f7fc9-98a3-4053-baaa-6f71d1d64271 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.668560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f7fc9-98a3-4053-baaa-6f71d1d64271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.668838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f7fc9-98a3-4053-baaa-6f71d1d64271 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.669556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f7fc9-98a3-4053-baaa-6f71d1d64271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:48.671572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f7fc9-98a3-4053-baaa-6f71d1d64271 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:48.672375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01399cf-39d3-4db1-9cce-6a49d5298afa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:48.749942 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 762/1546] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:48.759941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f7fc9-98a3-4053-baaa-6f71d1d64271 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:48.760271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 785/1547] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:48.761321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01399cf-39d3-4db1-9cce-6a49d5298afa admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:48.892526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01399cf-39d3-4db1-9cce-6a49d5298afa admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:48.904418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f7fc9-98a3-4053-baaa-6f71d1d64271 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:48.908159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee22058-f504-4bdb-ba9e-3fe261183a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:48.909582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee22058-f504-4bdb-ba9e-3fe261183a87 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.910847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee22058-f504-4bdb-ba9e-3fe261183a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.911902 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee22058-f504-4bdb-ba9e-3fe261183a87 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.913186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee22058-f504-4bdb-ba9e-3fe261183a87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:48.916883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee22058-f504-4bdb-ba9e-3fe261183a87 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:48.919387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83177880-2948-4086-b22f-26675e8980d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:48.920091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83177880-2948-4086-b22f-26675e8980d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.920388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83177880-2948-4086-b22f-26675e8980d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.920678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83177880-2948-4086-b22f-26675e8980d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.921730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83177880-2948-4086-b22f-26675e8980d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:48.922715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-83177880-2948-4086-b22f-26675e8980d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:13:48.922715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 22 11:13:48.922715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: Jul 22 11:13:48.923980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83177880-2948-4086-b22f-26675e8980d5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:48.945053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 786/1548] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:48.946709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978e9d1-e415-46a3-a4d5-c6b7ebce6dd1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.947289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978e9d1-e415-46a3-a4d5-c6b7ebce6dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.947654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978e9d1-e415-46a3-a4d5-c6b7ebce6dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.953697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 763/1549] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:48.955098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4b9d7-8d79-4a56-9550-f40ef266a495 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.955505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4b9d7-8d79-4a56-9550-f40ef266a495 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.955792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4b9d7-8d79-4a56-9550-f40ef266a495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.967534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2978e9d1-e415-46a3-a4d5-c6b7ebce6dd1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:48.968852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2978e9d1-e415-46a3-a4d5-c6b7ebce6dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:48.974679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8dc4b9d7-8d79-4a56-9550-f40ef266a495 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:48.976048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8dc4b9d7-8d79-4a56-9550-f40ef266a495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:48.976997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 787/1550] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:48.978218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59981f4-618f-4351-879a-3f9919eb042b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.978512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59981f4-618f-4351-879a-3f9919eb042b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.978801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59981f4-618f-4351-879a-3f9919eb042b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:48.990310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 764/1551] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:48.991645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e40c5a-3c03-41ae-8ced-9345d29acea3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:48.991940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e40c5a-3c03-41ae-8ced-9345d29acea3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:48.992445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e40c5a-3c03-41ae-8ced-9345d29acea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.001922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d59981f4-618f-4351-879a-3f9919eb042b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.003810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d59981f4-618f-4351-879a-3f9919eb042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.011923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 788/1552] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.012756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-88e40c5a-3c03-41ae-8ced-9345d29acea3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.013307 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60505a-6580-47ca-953e-7401563130dc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.013607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60505a-6580-47ca-953e-7401563130dc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.013891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60505a-6580-47ca-953e-7401563130dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.014003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-88e40c5a-3c03-41ae-8ced-9345d29acea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.023047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 765/1553] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.023880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e40c5a-3c03-41ae-8ced-9345d29acea3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.034048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e40c5a-3c03-41ae-8ced-9345d29acea3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.035207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ed60505a-6580-47ca-953e-7401563130dc None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.036442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ed60505a-6580-47ca-953e-7401563130dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.039201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d5168-6b38-4c6e-a217-55d05beb7587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.039897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d5168-6b38-4c6e-a217-55d05beb7587 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.040186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d5168-6b38-4c6e-a217-55d05beb7587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.040462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d5168-6b38-4c6e-a217-55d05beb7587 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.041188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d5168-6b38-4c6e-a217-55d05beb7587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.042823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d5168-6b38-4c6e-a217-55d05beb7587 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.044404 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 789/1554] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.045604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124123-c5df-4d28-a109-61637b96b15d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.045894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124123-c5df-4d28-a109-61637b96b15d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.046174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124123-c5df-4d28-a109-61637b96b15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.054283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 766/1555] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:49.055160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d5168-6b38-4c6e-a217-55d05beb7587 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.065051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d5168-6b38-4c6e-a217-55d05beb7587 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.066665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-97124123-c5df-4d28-a109-61637b96b15d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.067903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-97124123-c5df-4d28-a109-61637b96b15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.070759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bec31-0c82-4d19-947d-791347544603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.071448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bec31-0c82-4d19-947d-791347544603 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.071736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bec31-0c82-4d19-947d-791347544603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.072013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bec31-0c82-4d19-947d-791347544603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.072736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bec31-0c82-4d19-947d-791347544603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.074315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bec31-0c82-4d19-947d-791347544603 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.076650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 790/1556] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.077872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adbee3-0ee6-4c31-84ba-e8b4dd5b1136 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.078176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adbee3-0ee6-4c31-84ba-e8b4dd5b1136 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.078445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adbee3-0ee6-4c31-84ba-e8b4dd5b1136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.085659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 767/1557] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:49.086992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1742197-febd-42c2-8e27-218d4c8761de None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.087301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1742197-febd-42c2-8e27-218d4c8761de None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.087582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1742197-febd-42c2-8e27-218d4c8761de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.104593 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b6adbee3-0ee6-4c31-84ba-e8b4dd5b1136 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.105639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b1742197-febd-42c2-8e27-218d4c8761de None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.105794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b6adbee3-0ee6-4c31-84ba-e8b4dd5b1136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.106815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b1742197-febd-42c2-8e27-218d4c8761de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.114263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 791/1558] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.115543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1748a7-5ad7-4f31-964f-5e7f56234ab9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.115769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 768/1559] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.115877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1748a7-5ad7-4f31-964f-5e7f56234ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.116113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1748a7-5ad7-4f31-964f-5e7f56234ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.116989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090da02c-afa1-4576-bc16-456146b47e7b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.117294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090da02c-afa1-4576-bc16-456146b47e7b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.117573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090da02c-afa1-4576-bc16-456146b47e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.137297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7a1748a7-5ad7-4f31-964f-5e7f56234ab9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.138530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7a1748a7-5ad7-4f31-964f-5e7f56234ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.142243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-090da02c-afa1-4576-bc16-456146b47e7b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.144189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-090da02c-afa1-4576-bc16-456146b47e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.147239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 792/1560] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.148499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce2a24-646a-4b27-9003-f8d9b12d2b56 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.148790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce2a24-646a-4b27-9003-f8d9b12d2b56 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.149071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce2a24-646a-4b27-9003-f8d9b12d2b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.154567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 769/1561] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.155531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090da02c-afa1-4576-bc16-456146b47e7b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.165263 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090da02c-afa1-4576-bc16-456146b47e7b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.169949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c2c07-97d7-4a9e-934b-f023b4b2ca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.170750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c2c07-97d7-4a9e-934b-f023b4b2ca86 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.171041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c2c07-97d7-4a9e-934b-f023b4b2ca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.171347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c2c07-97d7-4a9e-934b-f023b4b2ca86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.172147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c2c07-97d7-4a9e-934b-f023b4b2ca86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.173725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c2c07-97d7-4a9e-934b-f023b4b2ca86 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.174396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-51ce2a24-646a-4b27-9003-f8d9b12d2b56 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.175780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-51ce2a24-646a-4b27-9003-f8d9b12d2b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.185999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 793/1562] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.189820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 770/1563] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:49.189960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aee90-068e-4991-ad8a-7c9d6e25c8aa None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.189960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aee90-068e-4991-ad8a-7c9d6e25c8aa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.189960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aee90-068e-4991-ad8a-7c9d6e25c8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.190215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c2c07-97d7-4a9e-934b-f023b4b2ca86 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.210303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c2c07-97d7-4a9e-934b-f023b4b2ca86 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.212860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-794aee90-068e-4991-ad8a-7c9d6e25c8aa None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.214083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-794aee90-068e-4991-ad8a-7c9d6e25c8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.215706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226171a-b8d8-4527-9603-e4f9d46637f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.216383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226171a-b8d8-4527-9603-e4f9d46637f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.216672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226171a-b8d8-4527-9603-e4f9d46637f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.216952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226171a-b8d8-4527-9603-e4f9d46637f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.217682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226171a-b8d8-4527-9603-e4f9d46637f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.219340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226171a-b8d8-4527-9603-e4f9d46637f5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.222601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 794/1564] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.223203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aee90-068e-4991-ad8a-7c9d6e25c8aa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.234343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 771/1565] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:49.234343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226171a-b8d8-4527-9603-e4f9d46637f5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.234625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aee90-068e-4991-ad8a-7c9d6e25c8aa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.237686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f465cc2-4bc3-4dec-b060-02228703678b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.238350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f465cc2-4bc3-4dec-b060-02228703678b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.238635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f465cc2-4bc3-4dec-b060-02228703678b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.239179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f465cc2-4bc3-4dec-b060-02228703678b admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.239936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f465cc2-4bc3-4dec-b060-02228703678b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:13:49.240244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f465cc2-4bc3-4dec-b060-02228703678b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.241844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f465cc2-4bc3-4dec-b060-02228703678b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.244519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226171a-b8d8-4527-9603-e4f9d46637f5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.249160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee7835-1e51-4fc0-8773-30b3871ec136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.249932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee7835-1e51-4fc0-8773-30b3871ec136 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.250219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee7835-1e51-4fc0-8773-30b3871ec136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.250506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee7835-1e51-4fc0-8773-30b3871ec136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.251237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee7835-1e51-4fc0-8773-30b3871ec136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:13:49.251587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee7835-1e51-4fc0-8773-30b3871ec136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.253711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee7835-1e51-4fc0-8773-30b3871ec136 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.254042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 795/1566] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:13:49.254851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f465cc2-4bc3-4dec-b060-02228703678b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.266133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f465cc2-4bc3-4dec-b060-02228703678b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.271071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 772/1567] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:13:49.271071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee7835-1e51-4fc0-8773-30b3871ec136 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.277301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14067047-4eaa-4cc3-9cfe-ed7701f7a8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.277965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14067047-4eaa-4cc3-9cfe-ed7701f7a8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.278254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14067047-4eaa-4cc3-9cfe-ed7701f7a8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.278540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14067047-4eaa-4cc3-9cfe-ed7701f7a8c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.279279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14067047-4eaa-4cc3-9cfe-ed7701f7a8c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:13:49.279634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14067047-4eaa-4cc3-9cfe-ed7701f7a8c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.280863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee7835-1e51-4fc0-8773-30b3871ec136 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.281220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14067047-4eaa-4cc3-9cfe-ed7701f7a8c2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.285463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41414a9-3cdc-4a94-841f-7f67b03d2772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.286104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41414a9-3cdc-4a94-841f-7f67b03d2772 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.286396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41414a9-3cdc-4a94-841f-7f67b03d2772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.286668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41414a9-3cdc-4a94-841f-7f67b03d2772 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.287413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41414a9-3cdc-4a94-841f-7f67b03d2772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:13:49.287722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41414a9-3cdc-4a94-841f-7f67b03d2772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.289295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41414a9-3cdc-4a94-841f-7f67b03d2772 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.291673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:49.294260 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 796/1568] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:13:49.295139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14067047-4eaa-4cc3-9cfe-ed7701f7a8c2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.305109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14067047-4eaa-4cc3-9cfe-ed7701f7a8c2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.311187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 773/1569] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:13:49.311187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41414a9-3cdc-4a94-841f-7f67b03d2772 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.314507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2249a6-36e1-433b-a699-c4ef333f7fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.315205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2249a6-36e1-433b-a699-c4ef333f7fed admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.316063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2249a6-36e1-433b-a699-c4ef333f7fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.316347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2249a6-36e1-433b-a699-c4ef333f7fed admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.317112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2249a6-36e1-433b-a699-c4ef333f7fed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:13:49.317425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2249a6-36e1-433b-a699-c4ef333f7fed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.318987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41414a9-3cdc-4a94-841f-7f67b03d2772 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.319096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2249a6-36e1-433b-a699-c4ef333f7fed admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.323595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3c804-6159-4713-ba31-8af570054b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.324457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3c804-6159-4713-ba31-8af570054b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.324740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3c804-6159-4713-ba31-8af570054b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.325016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3c804-6159-4713-ba31-8af570054b41 admin admin] PATH_INFO: `/v3/projects/28836b7cb9c646aaaa1bb1c1fae68128/users/691877dce1024aeda61629073d74602b/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.328102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:49.333602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 797/1570] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:13:49.334986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2249a6-36e1-433b-a699-c4ef333f7fed admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.342811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3c804-6159-4713-ba31-8af570054b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28836b7cb9c646aaaa1bb1c1fae68128, user_id=691877dce1024aeda61629073d74602b, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.345521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3c804-6159-4713-ba31-8af570054b41 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.345959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2249a6-36e1-433b-a699-c4ef333f7fed admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.351407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bd3b8-8243-478c-9142-18e58271dee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.352144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bd3b8-8243-478c-9142-18e58271dee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.352447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bd3b8-8243-478c-9142-18e58271dee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.352709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bd3b8-8243-478c-9142-18e58271dee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.353614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bd3b8-8243-478c-9142-18e58271dee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:13:49.353859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bd3b8-8243-478c-9142-18e58271dee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.355458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bd3b8-8243-478c-9142-18e58271dee8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.359126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 774/1571] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:49 2022] PUT /identity/v3/projects/28836b7cb9c646aaaa1bb1c1fae68128/users/691877dce1024aeda61629073d74602b/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:49.362997 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3c804-6159-4713-ba31-8af570054b41 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.368935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 798/1572] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:13:49.369913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bd3b8-8243-478c-9142-18e58271dee8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.381837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bd3b8-8243-478c-9142-18e58271dee8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.431039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3c804-6159-4713-ba31-8af570054b41 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.435508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c67cc-861b-49b7-84c2-5d912e8db0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.436384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c67cc-861b-49b7-84c2-5d912e8db0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.436664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c67cc-861b-49b7-84c2-5d912e8db0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.436938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c67cc-861b-49b7-84c2-5d912e8db0b9 admin admin] PATH_INFO: `/v3/projects/08cd35ed100244fdab3228d1afe62400/users/5202e2bdba064c4aaa84e15e1f0f0dea/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.438525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f13f98-e9ef-4206-9440-9ba9b6e026a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.439435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f13f98-e9ef-4206-9440-9ba9b6e026a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.439724 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f13f98-e9ef-4206-9440-9ba9b6e026a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.440008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f13f98-e9ef-4206-9440-9ba9b6e026a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.441303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f13f98-e9ef-4206-9440-9ba9b6e026a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.443248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f13f98-e9ef-4206-9440-9ba9b6e026a9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.451981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c67cc-861b-49b7-84c2-5d912e8db0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08cd35ed100244fdab3228d1afe62400, user_id=5202e2bdba064c4aaa84e15e1f0f0dea, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.454667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c67cc-861b-49b7-84c2-5d912e8db0b9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.468855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 775/1573] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:49 2022] PUT /identity/v3/projects/08cd35ed100244fdab3228d1afe62400/users/5202e2bdba064c4aaa84e15e1f0f0dea/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:49.469071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 799/1574] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.469729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f13f98-e9ef-4206-9440-9ba9b6e026a9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.470148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c67cc-861b-49b7-84c2-5d912e8db0b9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.534332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c67cc-861b-49b7-84c2-5d912e8db0b9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.536214 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f13f98-e9ef-4206-9440-9ba9b6e026a9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.539185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bc230-08b3-4b4d-b8bd-44eb88543fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.540023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bc230-08b3-4b4d-b8bd-44eb88543fcb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.540323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bc230-08b3-4b4d-b8bd-44eb88543fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.540633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bc230-08b3-4b4d-b8bd-44eb88543fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.540818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0c3f6-3df2-4e57-96cf-07a6fc19aafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.541337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bc230-08b3-4b4d-b8bd-44eb88543fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.541514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0c3f6-3df2-4e57-96cf-07a6fc19aafd admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.541787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0c3f6-3df2-4e57-96cf-07a6fc19aafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.542068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0c3f6-3df2-4e57-96cf-07a6fc19aafd admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.542809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0c3f6-3df2-4e57-96cf-07a6fc19aafd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.543546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bc230-08b3-4b4d-b8bd-44eb88543fcb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.544776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0c3f6-3df2-4e57-96cf-07a6fc19aafd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.575567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 800/1575] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.575886 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 776/1576] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.576390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0c3f6-3df2-4e57-96cf-07a6fc19aafd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.577245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bc230-08b3-4b4d-b8bd-44eb88543fcb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.622435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:49.640245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:49.643929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bc230-08b3-4b4d-b8bd-44eb88543fcb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.644219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0c3f6-3df2-4e57-96cf-07a6fc19aafd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.649134 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d35a6-cde2-4038-b4db-72b2deafb45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.649800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d35a6-cde2-4038-b4db-72b2deafb45f admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.649961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca44175-f405-4d5b-91a0-83adef8b1bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.650154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d35a6-cde2-4038-b4db-72b2deafb45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.650442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d35a6-cde2-4038-b4db-72b2deafb45f admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.650624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca44175-f405-4d5b-91a0-83adef8b1bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.650912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca44175-f405-4d5b-91a0-83adef8b1bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.651138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d35a6-cde2-4038-b4db-72b2deafb45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.651234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca44175-f405-4d5b-91a0-83adef8b1bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.651979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca44175-f405-4d5b-91a0-83adef8b1bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.653095 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d35a6-cde2-4038-b4db-72b2deafb45f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.653936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca44175-f405-4d5b-91a0-83adef8b1bbf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.681737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 801/1577] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.681907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 777/1578] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.682654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d35a6-cde2-4038-b4db-72b2deafb45f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.683272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4e2cc-754f-4487-9b24-8c6a73f7b71d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.683630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4e2cc-754f-4487-9b24-8c6a73f7b71d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.684024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4e2cc-754f-4487-9b24-8c6a73f7b71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.702100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-67d4e2cc-754f-4487-9b24-8c6a73f7b71d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.703353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-67d4e2cc-754f-4487-9b24-8c6a73f7b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.748274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d35a6-cde2-4038-b4db-72b2deafb45f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.753817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd65c3a-11db-40ef-93ff-6da64bb0ef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.754473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd65c3a-11db-40ef-93ff-6da64bb0ef62 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.754764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd65c3a-11db-40ef-93ff-6da64bb0ef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.755035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd65c3a-11db-40ef-93ff-6da64bb0ef62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.755770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd65c3a-11db-40ef-93ff-6da64bb0ef62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.757683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd65c3a-11db-40ef-93ff-6da64bb0ef62 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.763233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 802/1579] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.764196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4e2cc-754f-4487-9b24-8c6a73f7b71d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.773797 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4e2cc-754f-4487-9b24-8c6a73f7b71d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.779469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd0eaf-6ce7-4dfe-85a7-ac4dc5aa56d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.780194 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd0eaf-6ce7-4dfe-85a7-ac4dc5aa56d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.780492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd0eaf-6ce7-4dfe-85a7-ac4dc5aa56d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.780772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd0eaf-6ce7-4dfe-85a7-ac4dc5aa56d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.781483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd0eaf-6ce7-4dfe-85a7-ac4dc5aa56d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.783423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd0eaf-6ce7-4dfe-85a7-ac4dc5aa56d1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.788799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 778/1580] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.788799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114591fc-a720-424c-82fd-9313ee3b5a96 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.788799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114591fc-a720-424c-82fd-9313ee3b5a96 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.788799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114591fc-a720-424c-82fd-9313ee3b5a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.804826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 803/1581] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.805630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd0eaf-6ce7-4dfe-85a7-ac4dc5aa56d1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.809556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-114591fc-a720-424c-82fd-9313ee3b5a96 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:49.810768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-114591fc-a720-424c-82fd-9313ee3b5a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:49.815208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd0eaf-6ce7-4dfe-85a7-ac4dc5aa56d1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.871551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0a9a2-6beb-4c6e-8e17-9b1a57f8c6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.872138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 779/1582] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:49.872349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0a9a2-6beb-4c6e-8e17-9b1a57f8c6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.872708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0a9a2-6beb-4c6e-8e17-9b1a57f8c6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.872814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114591fc-a720-424c-82fd-9313ee3b5a96 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.872998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0a9a2-6beb-4c6e-8e17-9b1a57f8c6f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.873746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0a9a2-6beb-4c6e-8e17-9b1a57f8c6f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.880528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0a9a2-6beb-4c6e-8e17-9b1a57f8c6f8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.882269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114591fc-a720-424c-82fd-9313ee3b5a96 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.886798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56de5c-a2b0-4dca-9106-87288e688e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.887497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56de5c-a2b0-4dca-9106-87288e688e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.887784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56de5c-a2b0-4dca-9106-87288e688e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.888279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56de5c-a2b0-4dca-9106-87288e688e02 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.888756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56de5c-a2b0-4dca-9106-87288e688e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.890994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56de5c-a2b0-4dca-9106-87288e688e02 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.902302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 804/1583] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.903415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0a9a2-6beb-4c6e-8e17-9b1a57f8c6f8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.913207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 780/1584] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/users => generated 285 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.914004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56de5c-a2b0-4dca-9106-87288e688e02 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.914473 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0a9a2-6beb-4c6e-8e17-9b1a57f8c6f8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.919520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37ed11-d54e-41f8-9160-f9c1f4abbb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.920181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37ed11-d54e-41f8-9160-f9c1f4abbb60 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.920468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37ed11-d54e-41f8-9160-f9c1f4abbb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.920748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37ed11-d54e-41f8-9160-f9c1f4abbb60 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.921430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37ed11-d54e-41f8-9160-f9c1f4abbb60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.923397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37ed11-d54e-41f8-9160-f9c1f4abbb60 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.923661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56de5c-a2b0-4dca-9106-87288e688e02 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.929062 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad9238-5a12-4b59-b057-2ac1a7c15dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.929709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad9238-5a12-4b59-b057-2ac1a7c15dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.930001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad9238-5a12-4b59-b057-2ac1a7c15dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.930417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad9238-5a12-4b59-b057-2ac1a7c15dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.930963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad9238-5a12-4b59-b057-2ac1a7c15dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.932986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad9238-5a12-4b59-b057-2ac1a7c15dd5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.944753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 805/1585] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/users => generated 296 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.945751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37ed11-d54e-41f8-9160-f9c1f4abbb60 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.957716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 781/1586] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:49.957716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad9238-5a12-4b59-b057-2ac1a7c15dd5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:49.964972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad9238-5a12-4b59-b057-2ac1a7c15dd5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:49.969753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad3f68-6937-4cce-907b-bf64f9520796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:49.970407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad3f68-6937-4cce-907b-bf64f9520796 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:49.970725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad3f68-6937-4cce-907b-bf64f9520796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:49.971004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad3f68-6937-4cce-907b-bf64f9520796 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:49.972212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad3f68-6937-4cce-907b-bf64f9520796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:49.974131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad3f68-6937-4cce-907b-bf64f9520796 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:49.989823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:49.990321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:50.001685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 782/1587] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:49 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:50.003372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad3f68-6937-4cce-907b-bf64f9520796 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.006350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37ed11-d54e-41f8-9160-f9c1f4abbb60 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.010031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4da93-202d-4346-847b-8779f8288503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.010688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4da93-202d-4346-847b-8779f8288503 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.011003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4da93-202d-4346-847b-8779f8288503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.011313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4da93-202d-4346-847b-8779f8288503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.013332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad3f68-6937-4cce-907b-bf64f9520796 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.018023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e461089-9005-4031-aa96-794bf16f58c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.018708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e461089-9005-4031-aa96-794bf16f58c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.018998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e461089-9005-4031-aa96-794bf16f58c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.019306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e461089-9005-4031-aa96-794bf16f58c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.020041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e461089-9005-4031-aa96-794bf16f58c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.020522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4da93-202d-4346-847b-8779f8288503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.021640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e461089-9005-4031-aa96-794bf16f58c1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.021981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4da93-202d-4346-847b-8779f8288503 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.031911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 783/1588] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.032351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 806/1589] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:50.036450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e461089-9005-4031-aa96-794bf16f58c1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.036657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4da93-202d-4346-847b-8779f8288503 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.046395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4da93-202d-4346-847b-8779f8288503 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.048399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e461089-9005-4031-aa96-794bf16f58c1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.052578 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfeaf7-05fc-4425-8d61-1ae10324f451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.053320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfeaf7-05fc-4425-8d61-1ae10324f451 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.053897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfeaf7-05fc-4425-8d61-1ae10324f451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.054026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e9bab-7d77-44f3-99c7-8c6f24c5d984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.054338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfeaf7-05fc-4425-8d61-1ae10324f451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.054795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e9bab-7d77-44f3-99c7-8c6f24c5d984 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.055286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e9bab-7d77-44f3-99c7-8c6f24c5d984 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.055401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfeaf7-05fc-4425-8d61-1ae10324f451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.055584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e9bab-7d77-44f3-99c7-8c6f24c5d984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.057231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfeaf7-05fc-4425-8d61-1ae10324f451 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.066120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e9bab-7d77-44f3-99c7-8c6f24c5d984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.067610 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e9bab-7d77-44f3-99c7-8c6f24c5d984 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.069905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 807/1590] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.072739 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfeaf7-05fc-4425-8d61-1ae10324f451 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.078023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 784/1591] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:50.078900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e9bab-7d77-44f3-99c7-8c6f24c5d984 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.084156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfeaf7-05fc-4425-8d61-1ae10324f451 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.088699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e9bab-7d77-44f3-99c7-8c6f24c5d984 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.089480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f639a9-e118-433f-8dbd-f9c176ded40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.090520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f639a9-e118-433f-8dbd-f9c176ded40b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.090822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f639a9-e118-433f-8dbd-f9c176ded40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.091108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f639a9-e118-433f-8dbd-f9c176ded40b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.091936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f639a9-e118-433f-8dbd-f9c176ded40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.093285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb83310-39e8-4a87-97ab-4781787baf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.093629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f639a9-e118-433f-8dbd-f9c176ded40b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.093939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb83310-39e8-4a87-97ab-4781787baf20 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.094225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb83310-39e8-4a87-97ab-4781787baf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.094502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb83310-39e8-4a87-97ab-4781787baf20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.095291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb83310-39e8-4a87-97ab-4781787baf20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.097317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb83310-39e8-4a87-97ab-4781787baf20 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.105143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 808/1592] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.108361 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f639a9-e118-433f-8dbd-f9c176ded40b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.111422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 785/1593] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.111422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb83310-39e8-4a87-97ab-4781787baf20 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.120545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb83310-39e8-4a87-97ab-4781787baf20 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.121394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f639a9-e118-433f-8dbd-f9c176ded40b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.126172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34d54a-2fad-4750-b053-0d830cc43a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.126897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34d54a-2fad-4750-b053-0d830cc43a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.127188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34d54a-2fad-4750-b053-0d830cc43a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.127437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1450e3-19b5-4aeb-bea2-10c74619b198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.127584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34d54a-2fad-4750-b053-0d830cc43a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.128266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34d54a-2fad-4750-b053-0d830cc43a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.128378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1450e3-19b5-4aeb-bea2-10c74619b198 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.128746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1450e3-19b5-4aeb-bea2-10c74619b198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.129031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1450e3-19b5-4aeb-bea2-10c74619b198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.129860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1450e3-19b5-4aeb-bea2-10c74619b198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.129978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34d54a-2fad-4750-b053-0d830cc43a9d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.131877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1450e3-19b5-4aeb-bea2-10c74619b198 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.147654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 786/1594] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.147654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34d54a-2fad-4750-b053-0d830cc43a9d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.148070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 809/1595] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.148070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1450e3-19b5-4aeb-bea2-10c74619b198 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.160058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1450e3-19b5-4aeb-bea2-10c74619b198 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.160964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34d54a-2fad-4750-b053-0d830cc43a9d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.165387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc933a3-6a0f-415c-b71f-cb2f7814b52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.166363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc933a3-6a0f-415c-b71f-cb2f7814b52a admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.166528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc5bf2-077f-44f2-8378-68564e79e5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.166649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc933a3-6a0f-415c-b71f-cb2f7814b52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.166649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc933a3-6a0f-415c-b71f-cb2f7814b52a admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.166881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5bf2-077f-44f2-8378-68564e79e5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.167493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5bf2-077f-44f2-8378-68564e79e5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.167818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5bf2-077f-44f2-8378-68564e79e5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.167945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc933a3-6a0f-415c-b71f-cb2f7814b52a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.168478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc5bf2-077f-44f2-8378-68564e79e5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.170079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc933a3-6a0f-415c-b71f-cb2f7814b52a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.170368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc5bf2-077f-44f2-8378-68564e79e5b2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.183914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 810/1596] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.187268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 787/1597] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.188217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc933a3-6a0f-415c-b71f-cb2f7814b52a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.188351 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc5bf2-077f-44f2-8378-68564e79e5b2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.198635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc5bf2-077f-44f2-8378-68564e79e5b2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.198842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc933a3-6a0f-415c-b71f-cb2f7814b52a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.204327 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3819d-5aaf-4848-be31-c28292f207b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.204505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6b95a-2920-483a-a978-4b09dad000fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.205196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6b95a-2920-483a-a978-4b09dad000fa admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.205320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3819d-5aaf-4848-be31-c28292f207b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.205602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6b95a-2920-483a-a978-4b09dad000fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.205812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3819d-5aaf-4848-be31-c28292f207b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.205926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6b95a-2920-483a-a978-4b09dad000fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.206837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3819d-5aaf-4848-be31-c28292f207b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.207241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6b95a-2920-483a-a978-4b09dad000fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.208240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3819d-5aaf-4848-be31-c28292f207b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.208903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6b95a-2920-483a-a978-4b09dad000fa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.209610 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3819d-5aaf-4848-be31-c28292f207b3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.224199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 811/1598] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.224199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6b95a-2920-483a-a978-4b09dad000fa admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.224572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 788/1599] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.224924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3819d-5aaf-4848-be31-c28292f207b3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.238443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6b95a-2920-483a-a978-4b09dad000fa admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.241109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3819d-5aaf-4848-be31-c28292f207b3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.244615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26312ad4-bace-4a70-b3ca-9609190a1f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.245381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26312ad4-bace-4a70-b3ca-9609190a1f42 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.245672 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26312ad4-bace-4a70-b3ca-9609190a1f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.245955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26312ad4-bace-4a70-b3ca-9609190a1f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.246720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26312ad4-bace-4a70-b3ca-9609190a1f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.247632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426570a-0e23-4eac-918d-e85f8fb9a890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.248725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26312ad4-bace-4a70-b3ca-9609190a1f42 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.248852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426570a-0e23-4eac-918d-e85f8fb9a890 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.248852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426570a-0e23-4eac-918d-e85f8fb9a890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.249232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426570a-0e23-4eac-918d-e85f8fb9a890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.250319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426570a-0e23-4eac-918d-e85f8fb9a890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.253260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426570a-0e23-4eac-918d-e85f8fb9a890 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.261449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 812/1600] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.262341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26312ad4-bace-4a70-b3ca-9609190a1f42 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.272424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 789/1601] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:50.272424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426570a-0e23-4eac-918d-e85f8fb9a890 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.278305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26312ad4-bace-4a70-b3ca-9609190a1f42 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.282292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426570a-0e23-4eac-918d-e85f8fb9a890 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.284200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3c90c-534b-40f9-b6c7-ad5c429ead7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.285576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3c90c-534b-40f9-b6c7-ad5c429ead7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.285863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3c90c-534b-40f9-b6c7-ad5c429ead7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.286140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3c90c-534b-40f9-b6c7-ad5c429ead7d admin admin] PATH_INFO: `/v3/projects/f50a15db11214fb5a637fb400dcd928f/users/da75b78b267f47e8aaaafb41e3acb61a/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.287864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee4657-1cfc-42e8-824b-7e515bbdcaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.288835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee4657-1cfc-42e8-824b-7e515bbdcaf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.289270 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee4657-1cfc-42e8-824b-7e515bbdcaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.289770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee4657-1cfc-42e8-824b-7e515bbdcaf0 admin admin] PATH_INFO: `/v3/projects/761d03dc41b64911b761681bab034b36/users/372b0956ae414ec1bc8b1fdd2fcdcf47/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.304129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3c90c-534b-40f9-b6c7-ad5c429ead7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50a15db11214fb5a637fb400dcd928f, user_id=da75b78b267f47e8aaaafb41e3acb61a, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.305520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-33d3c90c-534b-40f9-b6c7-ad5c429ead7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:13:50.305520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 22 11:13:50.305520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: Jul 22 11:13:50.307798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3c90c-534b-40f9-b6c7-ad5c429ead7d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.308141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee4657-1cfc-42e8-824b-7e515bbdcaf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=761d03dc41b64911b761681bab034b36, user_id=372b0956ae414ec1bc8b1fdd2fcdcf47, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.311063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee4657-1cfc-42e8-824b-7e515bbdcaf0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.322542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 813/1602] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:50 2022] PUT /identity/v3/projects/f50a15db11214fb5a637fb400dcd928f/users/da75b78b267f47e8aaaafb41e3acb61a/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:50.323676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3c90c-534b-40f9-b6c7-ad5c429ead7d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.325529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 790/1603] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:50 2022] PUT /identity/v3/projects/761d03dc41b64911b761681bab034b36/users/372b0956ae414ec1bc8b1fdd2fcdcf47/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:50.328146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee4657-1cfc-42e8-824b-7e515bbdcaf0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.408654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3c90c-534b-40f9-b6c7-ad5c429ead7d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.414123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee4657-1cfc-42e8-824b-7e515bbdcaf0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.414856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058602f-1a0b-4eb8-b619-c367d09a557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.416146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058602f-1a0b-4eb8-b619-c367d09a557a admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.416568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058602f-1a0b-4eb8-b619-c367d09a557a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.416959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058602f-1a0b-4eb8-b619-c367d09a557a admin admin] PATH_INFO: `/v3/projects/c21697f0b1064161845b4ac2a0072f39/users/a7b94c3982cc49759cb9f95014dcf821/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.420229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43855-8a28-4779-b698-604f155dfcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.421128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43855-8a28-4779-b698-604f155dfcdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.421417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43855-8a28-4779-b698-604f155dfcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.421695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43855-8a28-4779-b698-604f155dfcdc admin admin] PATH_INFO: `/v3/projects/601d12851417478a8b39bbf0c9fcd03d/users/7c2c0270618240de8fd9290d01a7413a/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.436698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058602f-1a0b-4eb8-b619-c367d09a557a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21697f0b1064161845b4ac2a0072f39, user_id=a7b94c3982cc49759cb9f95014dcf821, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.439587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058602f-1a0b-4eb8-b619-c367d09a557a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.442748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e43855-8a28-4779-b698-604f155dfcdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=601d12851417478a8b39bbf0c9fcd03d, user_id=7c2c0270618240de8fd9290d01a7413a, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.445740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e43855-8a28-4779-b698-604f155dfcdc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.464324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 791/1604] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:50 2022] PUT /identity/v3/projects/601d12851417478a8b39bbf0c9fcd03d/users/7c2c0270618240de8fd9290d01a7413a/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:50.465267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 814/1605] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:50 2022] PUT /identity/v3/projects/c21697f0b1064161845b4ac2a0072f39/users/a7b94c3982cc49759cb9f95014dcf821/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:50.466110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43855-8a28-4779-b698-604f155dfcdc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.466378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058602f-1a0b-4eb8-b619-c367d09a557a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.510894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:50.514119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:50.541607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43855-8a28-4779-b698-604f155dfcdc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.546487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50a5f3-c19e-42c1-8119-2d34ccf22454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.547497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50a5f3-c19e-42c1-8119-2d34ccf22454 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.547847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50a5f3-c19e-42c1-8119-2d34ccf22454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.548190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50a5f3-c19e-42c1-8119-2d34ccf22454 admin admin] PATH_INFO: `/v3/projects/a0ca7d833cb44d02aabc2d345df833f3/users/81cf75f532a44b27930a8da72700d21a/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.549619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058602f-1a0b-4eb8-b619-c367d09a557a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:50.554638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c26434-7bad-43ab-95fa-f1d0bd19502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:50.555679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c26434-7bad-43ab-95fa-f1d0bd19502a admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.556045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c26434-7bad-43ab-95fa-f1d0bd19502a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.556402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c26434-7bad-43ab-95fa-f1d0bd19502a admin admin] PATH_INFO: `/v3/projects/c54daeb26bc247049e61b4364036be0a/users/29fd5aa8f0ef479d9c7ff20dbb503e87/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.565018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50a5f3-c19e-42c1-8119-2d34ccf22454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ca7d833cb44d02aabc2d345df833f3, user_id=81cf75f532a44b27930a8da72700d21a, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.568401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50a5f3-c19e-42c1-8119-2d34ccf22454 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.574864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c26434-7bad-43ab-95fa-f1d0bd19502a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54daeb26bc247049e61b4364036be0a, user_id=29fd5aa8f0ef479d9c7ff20dbb503e87, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:50.577788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c26434-7bad-43ab-95fa-f1d0bd19502a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:50.586492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 792/1606] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:50 2022] PUT /identity/v3/projects/a0ca7d833cb44d02aabc2d345df833f3/users/81cf75f532a44b27930a8da72700d21a/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:50.587925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ddf2d-7053-4619-84b0-26aebc13a00d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.588345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ddf2d-7053-4619-84b0-26aebc13a00d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.588731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ddf2d-7053-4619-84b0-26aebc13a00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.607324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 815/1607] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:50 2022] PUT /identity/v3/projects/c54daeb26bc247049e61b4364036be0a/users/29fd5aa8f0ef479d9c7ff20dbb503e87/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:50.614853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7ba66-9db8-4e67-9391-92e3dc647b3b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.615438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7ba66-9db8-4e67-9391-92e3dc647b3b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.615895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d11ddf2d-7053-4619-84b0-26aebc13a00d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:50.616091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7ba66-9db8-4e67-9391-92e3dc647b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.617998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d11ddf2d-7053-4619-84b0-26aebc13a00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:50.647981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-23e7ba66-9db8-4e67-9391-92e3dc647b3b None None] MFA Rules not processed for user `372b0956ae414ec1bc8b1fdd2fcdcf47`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:50.649508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-23e7ba66-9db8-4e67-9391-92e3dc647b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:50.724531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 793/1608] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:50.726229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377cfe1-780c-4b35-b315-2ee09c199088 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.728052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377cfe1-780c-4b35-b315-2ee09c199088 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.728953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377cfe1-780c-4b35-b315-2ee09c199088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.755805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9377cfe1-780c-4b35-b315-2ee09c199088 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:50.757112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9377cfe1-780c-4b35-b315-2ee09c199088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:50.760280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 816/1609] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:50.762119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e367df5-a84f-41a1-810b-a6c0653b2c45 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.762685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e367df5-a84f-41a1-810b-a6c0653b2c45 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.763298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e367df5-a84f-41a1-810b-a6c0653b2c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.771930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 794/1610] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:50.775430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33d32d-130e-47a0-a046-ca6dabd7e9b0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.775690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33d32d-130e-47a0-a046-ca6dabd7e9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.776053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33d32d-130e-47a0-a046-ca6dabd7e9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.801341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7e367df5-a84f-41a1-810b-a6c0653b2c45 None None] MFA Rules not processed for user `a7b94c3982cc49759cb9f95014dcf821`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:50.803012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7e367df5-a84f-41a1-810b-a6c0653b2c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:50.843398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ee33d32d-130e-47a0-a046-ca6dabd7e9b0 None None] MFA Rules not processed for user `81cf75f532a44b27930a8da72700d21a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:50.844676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee33d32d-130e-47a0-a046-ca6dabd7e9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:50.906663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 817/1611] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:50.908632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54732eb6-19ef-4b05-be1e-f0551f4bad2b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:50.909725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54732eb6-19ef-4b05-be1e-f0551f4bad2b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:50.911431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54732eb6-19ef-4b05-be1e-f0551f4bad2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:50.954974 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-54732eb6-19ef-4b05-be1e-f0551f4bad2b None None] MFA Rules not processed for user `29fd5aa8f0ef479d9c7ff20dbb503e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:50.958761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-54732eb6-19ef-4b05-be1e-f0551f4bad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:50.988085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 795/1612] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:50.989721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33d32d-130e-47a0-a046-ca6dabd7e9b0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:50.998774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:51.024455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33d32d-130e-47a0-a046-ca6dabd7e9b0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:51.036686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc08cb-326a-42c6-8962-0acdb7670248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:51.038769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc08cb-326a-42c6-8962-0acdb7670248 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.039210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc08cb-326a-42c6-8962-0acdb7670248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.040473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc08cb-326a-42c6-8962-0acdb7670248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.041408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc08cb-326a-42c6-8962-0acdb7670248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:51.046531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc08cb-326a-42c6-8962-0acdb7670248 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:51.085693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 818/1613] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:51.086960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54732eb6-19ef-4b05-be1e-f0551f4bad2b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:51.088569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-54732eb6-19ef-4b05-be1e-f0551f4bad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:51.106124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 796/1614] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:50 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:51.107568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc08cb-326a-42c6-8962-0acdb7670248 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:51.161731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:51.182258 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54732eb6-19ef-4b05-be1e-f0551f4bad2b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:51.242686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc08cb-326a-42c6-8962-0acdb7670248 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:51.247635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724244cf-d87c-4a00-a341-16aeacd97ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:51.248479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724244cf-d87c-4a00-a341-16aeacd97ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.249471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724244cf-d87c-4a00-a341-16aeacd97ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.250925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724244cf-d87c-4a00-a341-16aeacd97ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.323799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78dfe6-0702-4dac-917c-c0acdb5ab613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:51.324726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78dfe6-0702-4dac-917c-c0acdb5ab613 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.325186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78dfe6-0702-4dac-917c-c0acdb5ab613 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.325517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78dfe6-0702-4dac-917c-c0acdb5ab613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.384844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724244cf-d87c-4a00-a341-16aeacd97ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:51.387720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724244cf-d87c-4a00-a341-16aeacd97ca8 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:51.404293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 797/1615] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:51.405211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724244cf-d87c-4a00-a341-16aeacd97ca8 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:51.419566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724244cf-d87c-4a00-a341-16aeacd97ca8 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:51.425048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72944952-325d-4d82-8bf2-68442289de08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:51.425730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72944952-325d-4d82-8bf2-68442289de08 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.426025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72944952-325d-4d82-8bf2-68442289de08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.426301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72944952-325d-4d82-8bf2-68442289de08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.427332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72944952-325d-4d82-8bf2-68442289de08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:51.430382 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72944952-325d-4d82-8bf2-68442289de08 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:51.431952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78dfe6-0702-4dac-917c-c0acdb5ab613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:51.433965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78dfe6-0702-4dac-917c-c0acdb5ab613 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:51.452908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 819/1616] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:51.456739 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9c393-1c8b-4eee-b351-af322d6905b2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.457145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9c393-1c8b-4eee-b351-af322d6905b2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.457264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9c393-1c8b-4eee-b351-af322d6905b2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.470314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 820/1617] 10.222.0.20 () {54 vars in 962 bytes} [Fri Jul 22 11:13:51 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:13:51.471576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9c393-1c8b-4eee-b351-af322d6905b2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:51.481880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 798/1618] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:51 2022] POST /identity/v3/projects => generated 369 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:51.483919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72944952-325d-4d82-8bf2-68442289de08 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:51.508323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9c393-1c8b-4eee-b351-af322d6905b2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:51.590008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:51.650839 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74500df-77f5-45c8-b4ba-43cd3296064f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:51.651483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72944952-325d-4d82-8bf2-68442289de08 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:51.651876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74500df-77f5-45c8-b4ba-43cd3296064f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.652269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74500df-77f5-45c8-b4ba-43cd3296064f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.652650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74500df-77f5-45c8-b4ba-43cd3296064f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.653571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74500df-77f5-45c8-b4ba-43cd3296064f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:51.656034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74500df-77f5-45c8-b4ba-43cd3296064f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:51.657609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5a0c4-6753-4711-9c52-4363597167a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:51.658353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5a0c4-6753-4711-9c52-4363597167a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.658653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5a0c4-6753-4711-9c52-4363597167a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.658935 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5a0c4-6753-4711-9c52-4363597167a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.659816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5a0c4-6753-4711-9c52-4363597167a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:51.663121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5a0c4-6753-4711-9c52-4363597167a3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:51.691591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 821/1619] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:13:51 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:51.692963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da218e8-0682-4d56-a19f-a1e32b4f0fdc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.693263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da218e8-0682-4d56-a19f-a1e32b4f0fdc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.693592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da218e8-0682-4d56-a19f-a1e32b4f0fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.708970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 799/1620] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:51 2022] POST /identity/v3/users => generated 295 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:51.709834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5a0c4-6753-4711-9c52-4363597167a3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:51.728832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5a0c4-6753-4711-9c52-4363597167a3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:51.756154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9da218e8-0682-4d56-a19f-a1e32b4f0fdc None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:51.758305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9da218e8-0682-4d56-a19f-a1e32b4f0fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:51.807774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:51.842472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3647c3-87d7-4851-ad18-36d2e6191071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:51.843689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3647c3-87d7-4851-ad18-36d2e6191071 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.844092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3647c3-87d7-4851-ad18-36d2e6191071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.844546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3647c3-87d7-4851-ad18-36d2e6191071 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.847169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3647c3-87d7-4851-ad18-36d2e6191071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:51.849551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3647c3-87d7-4851-ad18-36d2e6191071 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:51.892061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 800/1621] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:51 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:51.892900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3647c3-87d7-4851-ad18-36d2e6191071 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:51.916438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 822/1622] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Jul 22 11:13:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:51.917516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da218e8-0682-4d56-a19f-a1e32b4f0fdc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:51.931582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3647c3-87d7-4851-ad18-36d2e6191071 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:51.940227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a3e78-cfaf-47ff-9600-f64539fa8902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:51.941584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a3e78-cfaf-47ff-9600-f64539fa8902 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.942035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a3e78-cfaf-47ff-9600-f64539fa8902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.942388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a3e78-cfaf-47ff-9600-f64539fa8902 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.943344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a3e78-cfaf-47ff-9600-f64539fa8902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:51.944276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da218e8-0682-4d56-a19f-a1e32b4f0fdc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:51.945934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a3e78-cfaf-47ff-9600-f64539fa8902 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:51.960745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576d403-42f0-40e7-ac0a-5aec5e89bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:51.961856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576d403-42f0-40e7-ac0a-5aec5e89bb7c admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:51.962148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576d403-42f0-40e7-ac0a-5aec5e89bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:51.962427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576d403-42f0-40e7-ac0a-5aec5e89bb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:51.963616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576d403-42f0-40e7-ac0a-5aec5e89bb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:51.965653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576d403-42f0-40e7-ac0a-5aec5e89bb7c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:51.982408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 801/1623] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:13:51 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:13:51.983891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a3e78-cfaf-47ff-9600-f64539fa8902 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:51.990973 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 823/1624] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:51.991909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576d403-42f0-40e7-ac0a-5aec5e89bb7c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.002357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a3e78-cfaf-47ff-9600-f64539fa8902 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.009492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e08a6a-4e08-44e7-a3ee-6e805972b71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.010427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e08a6a-4e08-44e7-a3ee-6e805972b71b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.010982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e08a6a-4e08-44e7-a3ee-6e805972b71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.011601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e08a6a-4e08-44e7-a3ee-6e805972b71b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.013955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e08a6a-4e08-44e7-a3ee-6e805972b71b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.016456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e08a6a-4e08-44e7-a3ee-6e805972b71b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.026195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576d403-42f0-40e7-ac0a-5aec5e89bb7c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.040082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883c7b8-9732-4d63-be29-3aaa990db616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.041431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883c7b8-9732-4d63-be29-3aaa990db616 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.041431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883c7b8-9732-4d63-be29-3aaa990db616 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.041431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883c7b8-9732-4d63-be29-3aaa990db616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.042219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 802/1625] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:52.043530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e08a6a-4e08-44e7-a3ee-6e805972b71b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.065881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e08a6a-4e08-44e7-a3ee-6e805972b71b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.076406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9fbf-037c-49b6-a711-8ce5f6550f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.077852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9fbf-037c-49b6-a711-8ce5f6550f51 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.078415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9fbf-037c-49b6-a711-8ce5f6550f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.079155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9fbf-037c-49b6-a711-8ce5f6550f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.080478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f9fbf-037c-49b6-a711-8ce5f6550f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.083311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f9fbf-037c-49b6-a711-8ce5f6550f51 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.100966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 803/1626] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:52.102865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9fbf-037c-49b6-a711-8ce5f6550f51 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.122477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9fbf-037c-49b6-a711-8ce5f6550f51 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.131579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5851b-1576-4ecd-a3a5-f0d8bd0aedd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.133395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5851b-1576-4ecd-a3a5-f0d8bd0aedd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.133939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5851b-1576-4ecd-a3a5-f0d8bd0aedd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.134594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5851b-1576-4ecd-a3a5-f0d8bd0aedd1 admin admin] PATH_INFO: `/v3/projects/1aebb6b957bb4988b1e6dfdb335bbb89/users/8f133326a87744ce8d83b3fd44e8cc34/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.153980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883c7b8-9732-4d63-be29-3aaa990db616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.155339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883c7b8-9732-4d63-be29-3aaa990db616 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.157462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5851b-1576-4ecd-a3a5-f0d8bd0aedd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aebb6b957bb4988b1e6dfdb335bbb89, user_id=8f133326a87744ce8d83b3fd44e8cc34, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.161145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5851b-1576-4ecd-a3a5-f0d8bd0aedd1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.186402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 824/1627] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:52.190048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883c7b8-9732-4d63-be29-3aaa990db616 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.212326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 804/1628] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:52 2022] PUT /identity/v3/projects/1aebb6b957bb4988b1e6dfdb335bbb89/users/8f133326a87744ce8d83b3fd44e8cc34/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:52.213008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5851b-1576-4ecd-a3a5-f0d8bd0aedd1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.230614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883c7b8-9732-4d63-be29-3aaa990db616 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.275783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:52.321819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5851b-1576-4ecd-a3a5-f0d8bd0aedd1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.327412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c082877-b22d-4286-b4b8-92f1721a6822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.328708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c082877-b22d-4286-b4b8-92f1721a6822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.329259 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c082877-b22d-4286-b4b8-92f1721a6822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.329757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c082877-b22d-4286-b4b8-92f1721a6822 admin admin] PATH_INFO: `/v3/projects/ff247488a6914b45a0879bf4420d35fd/users/f8f05a002254415ea0ac81902635ed6e/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.347867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c082877-b22d-4286-b4b8-92f1721a6822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff247488a6914b45a0879bf4420d35fd, user_id=f8f05a002254415ea0ac81902635ed6e, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.352913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c082877-b22d-4286-b4b8-92f1721a6822 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.365696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2b159-0911-4555-8321-29ad9990af35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.366612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2b159-0911-4555-8321-29ad9990af35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.366899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2b159-0911-4555-8321-29ad9990af35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.367865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2b159-0911-4555-8321-29ad9990af35 admin admin] PATH_INFO: `/v3/projects/5aba579171904fd9a6babe788a549c87/users/9f3f9c10fbc64886a3da4633cad3e978/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.378966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 805/1629] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:52 2022] PUT /identity/v3/projects/ff247488a6914b45a0879bf4420d35fd/users/f8f05a002254415ea0ac81902635ed6e/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:52.380349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c082877-b22d-4286-b4b8-92f1721a6822 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.412161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2b159-0911-4555-8321-29ad9990af35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aba579171904fd9a6babe788a549c87, user_id=9f3f9c10fbc64886a3da4633cad3e978, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.415003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2b159-0911-4555-8321-29ad9990af35 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.448871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 825/1630] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:52 2022] PUT /identity/v3/projects/5aba579171904fd9a6babe788a549c87/users/9f3f9c10fbc64886a3da4633cad3e978/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:52.449779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2b159-0911-4555-8321-29ad9990af35 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.503767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:52.531151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c082877-b22d-4286-b4b8-92f1721a6822 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.572221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2b159-0911-4555-8321-29ad9990af35 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.588631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b639948-0e1d-464d-96ad-cb8a5a2b7afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.590154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b639948-0e1d-464d-96ad-cb8a5a2b7afd admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.590936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b639948-0e1d-464d-96ad-cb8a5a2b7afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.591433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b639948-0e1d-464d-96ad-cb8a5a2b7afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.592383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b639948-0e1d-464d-96ad-cb8a5a2b7afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.596090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b639948-0e1d-464d-96ad-cb8a5a2b7afd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.636152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 826/1631] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:52.636152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b639948-0e1d-464d-96ad-cb8a5a2b7afd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.665154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b639948-0e1d-464d-96ad-cb8a5a2b7afd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.672256 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89480200-3a6c-4e72-be7a-fb15e0e6e7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.673179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89480200-3a6c-4e72-be7a-fb15e0e6e7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.673637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89480200-3a6c-4e72-be7a-fb15e0e6e7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.674067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89480200-3a6c-4e72-be7a-fb15e0e6e7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.675649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89480200-3a6c-4e72-be7a-fb15e0e6e7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.676424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1f5fb-4444-4bfe-ac7c-6900dba8ba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.677470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1f5fb-4444-4bfe-ac7c-6900dba8ba7f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.679160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1f5fb-4444-4bfe-ac7c-6900dba8ba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.679160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1f5fb-4444-4bfe-ac7c-6900dba8ba7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.680407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89480200-3a6c-4e72-be7a-fb15e0e6e7e1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.681352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b1f5fb-4444-4bfe-ac7c-6900dba8ba7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.684100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b1f5fb-4444-4bfe-ac7c-6900dba8ba7f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.704712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 806/1632] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:52.704712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89480200-3a6c-4e72-be7a-fb15e0e6e7e1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.710938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 827/1633] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:13:52.710938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6657175-786b-46db-a773-a79cfc8177a5 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.710938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6657175-786b-46db-a773-a79cfc8177a5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.710938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6657175-786b-46db-a773-a79cfc8177a5 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.722748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 828/1634] 10.222.0.20 () {54 vars in 912 bytes} [Fri Jul 22 11:13:52 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:13:52.724880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6657175-786b-46db-a773-a79cfc8177a5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.728032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89480200-3a6c-4e72-be7a-fb15e0e6e7e1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.733113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1e92e-9fe9-438f-8e8e-2a8869cfdac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.734264 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1e92e-9fe9-438f-8e8e-2a8869cfdac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.734556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1e92e-9fe9-438f-8e8e-2a8869cfdac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.734837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1e92e-9fe9-438f-8e8e-2a8869cfdac8 admin admin] PATH_INFO: `/v3/projects/ff247488a6914b45a0879bf4420d35fd/users/f8f05a002254415ea0ac81902635ed6e/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.737003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1e92e-9fe9-438f-8e8e-2a8869cfdac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff247488a6914b45a0879bf4420d35fd, user_id=f8f05a002254415ea0ac81902635ed6e, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.740198 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1e92e-9fe9-438f-8e8e-2a8869cfdac8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.749317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6657175-786b-46db-a773-a79cfc8177a5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.762353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 807/1635] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:52 2022] PUT /identity/v3/projects/ff247488a6914b45a0879bf4420d35fd/users/f8f05a002254415ea0ac81902635ed6e/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:52.762353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1e92e-9fe9-438f-8e8e-2a8869cfdac8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:52.850350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1e92e-9fe9-438f-8e8e-2a8869cfdac8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:52.868807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78823a4f-da78-430e-abbd-92e6bc5c036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.869133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0c9b1-799a-4c60-a0c9-eee30cb96496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:52.870126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823a4f-da78-430e-abbd-92e6bc5c036e admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.870510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823a4f-da78-430e-abbd-92e6bc5c036e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.870885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823a4f-da78-430e-abbd-92e6bc5c036e admin admin] PATH_INFO: `/v3/projects/1aebb6b957bb4988b1e6dfdb335bbb89/users/8f133326a87744ce8d83b3fd44e8cc34/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.871135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0c9b1-799a-4c60-a0c9-eee30cb96496 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.871599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0c9b1-799a-4c60-a0c9-eee30cb96496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.872166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0c9b1-799a-4c60-a0c9-eee30cb96496 admin admin] PATH_INFO: `/v3/projects/5aba579171904fd9a6babe788a549c87/users/9f3f9c10fbc64886a3da4633cad3e978/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.873284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78823a4f-da78-430e-abbd-92e6bc5c036e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aebb6b957bb4988b1e6dfdb335bbb89, user_id=8f133326a87744ce8d83b3fd44e8cc34, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.876154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0c9b1-799a-4c60-a0c9-eee30cb96496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aba579171904fd9a6babe788a549c87, user_id=9f3f9c10fbc64886a3da4633cad3e978, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:52.876504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78823a4f-da78-430e-abbd-92e6bc5c036e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.882024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0c9b1-799a-4c60-a0c9-eee30cb96496 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:52.891992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 829/1636] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:52 2022] PUT /identity/v3/projects/1aebb6b957bb4988b1e6dfdb335bbb89/users/8f133326a87744ce8d83b3fd44e8cc34/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:52.904632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb26f9c-fe77-4df8-b2b8-ae085974ee3a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.905554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb26f9c-fe77-4df8-b2b8-ae085974ee3a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.905554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb26f9c-fe77-4df8-b2b8-ae085974ee3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.907989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 808/1637] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:13:52 2022] PUT /identity/v3/projects/5aba579171904fd9a6babe788a549c87/users/9f3f9c10fbc64886a3da4633cad3e978/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:13:52.919034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b960f-183e-4219-8944-fa6f6843a78c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:52.919920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b960f-183e-4219-8944-fa6f6843a78c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:52.920384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b960f-183e-4219-8944-fa6f6843a78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:52.970964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9fb26f9c-fe77-4df8-b2b8-ae085974ee3a None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:52.972615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9fb26f9c-fe77-4df8-b2b8-ae085974ee3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:53.010841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:53.013180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1f1b960f-183e-4219-8944-fa6f6843a78c None None] MFA Rules not processed for user `f8f05a002254415ea0ac81902635ed6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:53.019775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1f1b960f-183e-4219-8944-fa6f6843a78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:53.106617 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 830/1638] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:13:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:53.108265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa55e1-d2ad-4f7c-8ef5-7f9fdca0d977 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:53.108659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa55e1-d2ad-4f7c-8ef5-7f9fdca0d977 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:53.109149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa55e1-d2ad-4f7c-8ef5-7f9fdca0d977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:53.156371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-55aa55e1-d2ad-4f7c-8ef5-7f9fdca0d977 None None] MFA Rules not processed for user `8f133326a87744ce8d83b3fd44e8cc34`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:53.161563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-55aa55e1-d2ad-4f7c-8ef5-7f9fdca0d977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:53.191873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:53.280993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 809/1639] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:53.280993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca20c60-ce9b-447a-b0b7-742556aeaa4f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:53.280993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca20c60-ce9b-447a-b0b7-742556aeaa4f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:53.280993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca20c60-ce9b-447a-b0b7-742556aeaa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:53.323383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eca20c60-ce9b-447a-b0b7-742556aeaa4f None None] MFA Rules not processed for user `9f3f9c10fbc64886a3da4633cad3e978`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:53.326092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eca20c60-ce9b-447a-b0b7-742556aeaa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:53.377940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 831/1640] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:53.378838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa55e1-d2ad-4f7c-8ef5-7f9fdca0d977 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:53.480777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 810/1641] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:13:53 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:53.481764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca20c60-ce9b-447a-b0b7-742556aeaa4f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:53.488410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa55e1-d2ad-4f7c-8ef5-7f9fdca0d977 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:53.500299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b5b37-15f1-4323-80b8-24a2bb76126f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:53.501285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b5b37-15f1-4323-80b8-24a2bb76126f service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:53.501843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b5b37-15f1-4323-80b8-24a2bb76126f service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:53.502224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b5b37-15f1-4323-80b8-24a2bb76126f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:53.504335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca20c60-ce9b-447a-b0b7-742556aeaa4f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:53.513134 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0accf-bf89-48be-a018-accdcc55eac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:53.513486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0accf-bf89-48be-a018-accdcc55eac4 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:53.513779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0accf-bf89-48be-a018-accdcc55eac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:53.514066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0accf-bf89-48be-a018-accdcc55eac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:53.519726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b5b37-15f1-4323-80b8-24a2bb76126f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:53.521506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b5b37-15f1-4323-80b8-24a2bb76126f service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:53.535714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 832/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:53.545844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0accf-bf89-48be-a018-accdcc55eac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:53.548884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0accf-bf89-48be-a018-accdcc55eac4 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:53.581536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 811/1643] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:53.680505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b5b37-15f1-4323-80b8-24a2bb76126f service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:53.691948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b5b37-15f1-4323-80b8-24a2bb76126f service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:53.703659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c8b03-e055-47a6-b9bb-679b51b3984b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:53.704458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c8b03-e055-47a6-b9bb-679b51b3984b service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:53.706279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c8b03-e055-47a6-b9bb-679b51b3984b service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:53.707408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c8b03-e055-47a6-b9bb-679b51b3984b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:53.817487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:13:53.859366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c8b03-e055-47a6-b9bb-679b51b3984b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:53.860836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c8b03-e055-47a6-b9bb-679b51b3984b service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:53.887682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 833/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:54.408876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0accf-bf89-48be-a018-accdcc55eac4 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:54.424758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0accf-bf89-48be-a018-accdcc55eac4 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:54.438408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b8ac1-733e-4e33-94d3-faaaa2a80714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:54.439738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b8ac1-733e-4e33-94d3-faaaa2a80714 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:54.440905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b8ac1-733e-4e33-94d3-faaaa2a80714 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:54.443789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b8ac1-733e-4e33-94d3-faaaa2a80714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:54.466308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b8ac1-733e-4e33-94d3-faaaa2a80714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:54.470609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b8ac1-733e-4e33-94d3-faaaa2a80714 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:54.499876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 812/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:54 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:54.728058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c8b03-e055-47a6-b9bb-679b51b3984b service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:54.741644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c8b03-e055-47a6-b9bb-679b51b3984b service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:54.750437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f717e-904f-449e-b46f-84b180aa1469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:54.751490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f717e-904f-449e-b46f-84b180aa1469 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:54.751969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f717e-904f-449e-b46f-84b180aa1469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:54.752436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f717e-904f-449e-b46f-84b180aa1469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:54.890446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f717e-904f-449e-b46f-84b180aa1469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:54.894504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f717e-904f-449e-b46f-84b180aa1469 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:54.933278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 834/1646] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:54.976663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b8ac1-733e-4e33-94d3-faaaa2a80714 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:54.986597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b8ac1-733e-4e33-94d3-faaaa2a80714 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:54.999668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de7fd5-9fdb-4651-ba85-380abdf8e0f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:55.001401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de7fd5-9fdb-4651-ba85-380abdf8e0f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:55.001741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de7fd5-9fdb-4651-ba85-380abdf8e0f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:55.002027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de7fd5-9fdb-4651-ba85-380abdf8e0f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:55.152756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de7fd5-9fdb-4651-ba85-380abdf8e0f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:55.157556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de7fd5-9fdb-4651-ba85-380abdf8e0f9 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:55.203073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 813/1647] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:13:55.533100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab38248-610d-4d6c-9b83-4e9e89b01a62 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:55.533100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab38248-610d-4d6c-9b83-4e9e89b01a62 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:55.533419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab38248-610d-4d6c-9b83-4e9e89b01a62 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:55.535940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 835/1648] 10.222.0.20 () {54 vars in 962 bytes} [Fri Jul 22 11:13:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:13:55.553499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccebdea-9306-4289-8fe3-75ca30663cdd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:55.554235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccebdea-9306-4289-8fe3-75ca30663cdd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:55.555421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccebdea-9306-4289-8fe3-75ca30663cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:55.682530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cccebdea-9306-4289-8fe3-75ca30663cdd None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:55.685263 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cccebdea-9306-4289-8fe3-75ca30663cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:55.725702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 814/1649] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Jul 22 11:13:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:55.731818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e821511-f463-4602-9ac5-84d571efee16 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:55.732444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e821511-f463-4602-9ac5-84d571efee16 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:55.732552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e821511-f463-4602-9ac5-84d571efee16 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:55.736052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 836/1650] 10.222.0.20 () {54 vars in 912 bytes} [Fri Jul 22 11:13:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:13:55.743225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5aed0-173c-4dda-a7d1-48e223620904 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:55.743611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5aed0-173c-4dda-a7d1-48e223620904 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:55.744016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5aed0-173c-4dda-a7d1-48e223620904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:55.799746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c8c5aed0-173c-4dda-a7d1-48e223620904 None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:13:55.802699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8c5aed0-173c-4dda-a7d1-48e223620904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:13:55.836709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 815/1651] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:13:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:13:55.850773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e821511-f463-4602-9ac5-84d571efee16 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:13:55.866824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e821511-f463-4602-9ac5-84d571efee16 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:13:55.875420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517cbef-5b92-4231-be3e-4ae0aa06a545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:13:55.877438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517cbef-5b92-4231-be3e-4ae0aa06a545 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:13:55.879121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517cbef-5b92-4231-be3e-4ae0aa06a545 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:13:55.879606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517cbef-5b92-4231-be3e-4ae0aa06a545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:13:55.912510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517cbef-5b92-4231-be3e-4ae0aa06a545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:13:55.919295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517cbef-5b92-4231-be3e-4ae0aa06a545 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:13:55.985303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 837/1652] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:13:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:00.128391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5aed0-173c-4dda-a7d1-48e223620904 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:00.264953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:00.320815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5aed0-173c-4dda-a7d1-48e223620904 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:00.342353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:00.343541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:00.344115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:00.348298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] PATH_INFO: `/v3/users/a7b94c3982cc49759cb9f95014dcf821` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:00.352411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b94c3982cc49759cb9f95014dcf821)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:00.365141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:00.491997 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b94c3982cc49759cb9f95014dcf821'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:00.510262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b94c3982cc49759cb9f95014dcf821'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:00.530957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b94c3982cc49759cb9f95014dcf821'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:00.587156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b94c3982cc49759cb9f95014dcf821'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:00.588341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b94c3982cc49759cb9f95014dcf821'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:00.611726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 816/1653] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:00 2022] DELETE /identity/v3/users/a7b94c3982cc49759cb9f95014dcf821 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:00.619497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829cd19-324b-45da-b5cc-278c73e2f20b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:00.620139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829cd19-324b-45da-b5cc-278c73e2f20b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:00.621311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829cd19-324b-45da-b5cc-278c73e2f20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:00.685093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d829cd19-324b-45da-b5cc-278c73e2f20b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:00.687162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d829cd19-324b-45da-b5cc-278c73e2f20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:00.988039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:00.991170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:01.012864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 838/1654] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:01.098117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829cd19-324b-45da-b5cc-278c73e2f20b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:01.101837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d829cd19-324b-45da-b5cc-278c73e2f20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:01.156416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53e67-6b48-448e-bcb1-72114da2996e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:01.161830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:01.162521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:01.162812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:01.163098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:01.165214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:01.197159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829cd19-324b-45da-b5cc-278c73e2f20b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:01.209536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eaf3e7-4d37-4309-abf7-6b7888f8a5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:01.210669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eaf3e7-4d37-4309-abf7-6b7888f8a5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:01.211865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eaf3e7-4d37-4309-abf7-6b7888f8a5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:01.212549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eaf3e7-4d37-4309-abf7-6b7888f8a5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:01.229715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eaf3e7-4d37-4309-abf7-6b7888f8a5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:01.232563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eaf3e7-4d37-4309-abf7-6b7888f8a5d2 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:01.265487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 839/1655] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:01.277843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:01.279806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:01.295024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 817/1656] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:01.731828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477794f-4c5d-4aaf-b58e-64a26efbfeeb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:01.734745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477794f-4c5d-4aaf-b58e-64a26efbfeeb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:01.735345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477794f-4c5d-4aaf-b58e-64a26efbfeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:01.771731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:01.794079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5477794f-4c5d-4aaf-b58e-64a26efbfeeb None None] MFA Rules not processed for user `691877dce1024aeda61629073d74602b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:01.798264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5477794f-4c5d-4aaf-b58e-64a26efbfeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:02.050575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 840/1657] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:02.070057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:02.072138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:02.101124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:02.298106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2306c1-6f7f-4876-a9c7-47babb4acccf service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:02.318585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4870518-bd77-4601-8f32-f9a0365841c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:02.322014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4870518-bd77-4601-8f32-f9a0365841c9 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:02.324293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4870518-bd77-4601-8f32-f9a0365841c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:02.324550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4870518-bd77-4601-8f32-f9a0365841c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:02.349025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4870518-bd77-4601-8f32-f9a0365841c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:02.351552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4870518-bd77-4601-8f32-f9a0365841c9 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:02.391658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 818/1658] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:02.654290 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdbe2c-1412-4dd1-ab5f-6e245d6a5cb0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:02.655199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdbe2c-1412-4dd1-ab5f-6e245d6a5cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:02.655924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdbe2c-1412-4dd1-ab5f-6e245d6a5cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:02.768579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f1bdbe2c-1412-4dd1-ab5f-6e245d6a5cb0 None None] MFA Rules not processed for user `5202e2bdba064c4aaa84e15e1f0f0dea`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:02.770284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f1bdbe2c-1412-4dd1-ab5f-6e245d6a5cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:02.994156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 841/1659] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:02.996088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af7249-9944-4bbe-aec9-47c510aea3bb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:02.996392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af7249-9944-4bbe-aec9-47c510aea3bb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:02.996682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af7249-9944-4bbe-aec9-47c510aea3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:03.048641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-50af7249-9944-4bbe-aec9-47c510aea3bb None None] MFA Rules not processed for user `5202e2bdba064c4aaa84e15e1f0f0dea`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:03.050066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-50af7249-9944-4bbe-aec9-47c510aea3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:03.085835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 819/1660] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:03.093992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdbe2c-1412-4dd1-ab5f-6e245d6a5cb0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:03.116279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdbe2c-1412-4dd1-ab5f-6e245d6a5cb0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:03.129802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3419114-6e46-460f-9894-70c21121261b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:03.130504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3419114-6e46-460f-9894-70c21121261b service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:03.130798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3419114-6e46-460f-9894-70c21121261b service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:03.131080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3419114-6e46-460f-9894-70c21121261b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:03.165663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3419114-6e46-460f-9894-70c21121261b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:03.167407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3419114-6e46-460f-9894-70c21121261b service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:03.200856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 842/1661] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:03.263492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af7249-9944-4bbe-aec9-47c510aea3bb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:03.265485 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-50af7249-9944-4bbe-aec9-47c510aea3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:03.294439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af7249-9944-4bbe-aec9-47c510aea3bb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:03.302966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:03.304980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:03.305427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:03.305866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] PATH_INFO: `/v3/projects/c21697f0b1064161845b4ac2a0072f39` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:03.308130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21697f0b1064161845b4ac2a0072f39)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:03.309612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:14:03.309612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 22 11:14:03.309612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: Jul 22 11:14:03.312351 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:03.469581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21697f0b1064161845b4ac2a0072f39', 'request_id': 'req-753a837a-82d4-404d-9348-eb8e2894e370'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:03.489473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21697f0b1064161845b4ac2a0072f39', 'request_id': 'req-753a837a-82d4-404d-9348-eb8e2894e370'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:03.498431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21697f0b1064161845b4ac2a0072f39', 'request_id': 'req-753a837a-82d4-404d-9348-eb8e2894e370'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:03.529749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] The token cache is being invalidate because project c21697f0b1064161845b4ac2a0072f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:03.530151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:03.531146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-753a837a-82d4-404d-9348-eb8e2894e370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:03.536320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 820/1662] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:03 2022] DELETE /identity/v3/projects/c21697f0b1064161845b4ac2a0072f39 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:03.540378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3419114-6e46-460f-9894-70c21121261b service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:03.558266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3419114-6e46-460f-9894-70c21121261b service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:03.670244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:03.671514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:03.671809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:03.672094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] PATH_INFO: `/v3/users/9f3f9c10fbc64886a3da4633cad3e978` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:03.673434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3f9c10fbc64886a3da4633cad3e978)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:03.675099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 22 11:14:03.675099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 22 11:14:03.675099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: Jul 22 11:14:03.677450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:03.759459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:03.910846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3f9c10fbc64886a3da4633cad3e978'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:03.912555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3f9c10fbc64886a3da4633cad3e978'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:03.933244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3f9c10fbc64886a3da4633cad3e978'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:03.991686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3f9c10fbc64886a3da4633cad3e978'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:04.019674 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae6891-b11c-4e47-983e-c1a93a0765ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3f9c10fbc64886a3da4633cad3e978'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:04.034395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 843/1663] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:03 2022] DELETE /identity/v3/users/9f3f9c10fbc64886a3da4633cad3e978 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:04.107251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443258c7-c038-444d-8832-ebf795953586 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:04.107854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443258c7-c038-444d-8832-ebf795953586 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:04.109634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443258c7-c038-444d-8832-ebf795953586 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:04.111946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 821/1664] 10.222.0.20 () {54 vars in 912 bytes} [Fri Jul 22 11:14:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:14:04.120444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a4992-8f1d-43d9-a263-83b21e32fb07 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:04.120942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a4992-8f1d-43d9-a263-83b21e32fb07 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:04.121383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a4992-8f1d-43d9-a263-83b21e32fb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:04.261927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b98a4992-8f1d-43d9-a263-83b21e32fb07 None None] MFA Rules not processed for user `0a0ce1804b514b42845ab33c7bfee309`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:04.263532 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b98a4992-8f1d-43d9-a263-83b21e32fb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:04.435044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 844/1665] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:14:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:04.488511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443258c7-c038-444d-8832-ebf795953586 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:04.491337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-443258c7-c038-444d-8832-ebf795953586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:04.567890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443258c7-c038-444d-8832-ebf795953586 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:04.576643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653512f-2833-4683-994c-61f9baa4b3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:04.579187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653512f-2833-4683-994c-61f9baa4b3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:04.579810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653512f-2833-4683-994c-61f9baa4b3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:04.582112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653512f-2833-4683-994c-61f9baa4b3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:04.613611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653512f-2833-4683-994c-61f9baa4b3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:04.621071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653512f-2833-4683-994c-61f9baa4b3b6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:04.632205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 822/1666] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:05.267392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a4992-8f1d-43d9-a263-83b21e32fb07 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:05.269754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b98a4992-8f1d-43d9-a263-83b21e32fb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:05.436188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a4992-8f1d-43d9-a263-83b21e32fb07 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:05.454789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:05.456391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:05.456689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:05.463491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] PATH_INFO: `/v3/projects/5aba579171904fd9a6babe788a549c87` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:05.478553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aba579171904fd9a6babe788a549c87)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:05.484136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:05.602771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:05.634176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aba579171904fd9a6babe788a549c87', 'request_id': 'req-31c03b07-4990-4053-ba99-56e32f2bacac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:05.648841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aba579171904fd9a6babe788a549c87', 'request_id': 'req-31c03b07-4990-4053-ba99-56e32f2bacac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:05.657805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d73aed-94c2-47a9-bcfc-35738df5045b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:05.658383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d73aed-94c2-47a9-bcfc-35738df5045b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:05.658383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d73aed-94c2-47a9-bcfc-35738df5045b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:05.661261 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 823/1667] 10.222.0.20 () {54 vars in 961 bytes} [Fri Jul 22 11:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:14:05.669868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aba579171904fd9a6babe788a549c87', 'request_id': 'req-31c03b07-4990-4053-ba99-56e32f2bacac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:05.669999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080f6bd-50a0-4bdb-a0d2-27d7903112c3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:05.670227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080f6bd-50a0-4bdb-a0d2-27d7903112c3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:05.670471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080f6bd-50a0-4bdb-a0d2-27d7903112c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:05.707218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] The token cache is being invalidate because project 5aba579171904fd9a6babe788a549c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:05.708553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:05.711084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31c03b07-4990-4053-ba99-56e32f2bacac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:05.720419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 845/1668] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:05 2022] DELETE /identity/v3/projects/5aba579171904fd9a6babe788a549c87 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:05.724387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cab1a-5f9b-4c95-80a5-c82ac544c08f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:05.724924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cab1a-5f9b-4c95-80a5-c82ac544c08f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:05.725406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cab1a-5f9b-4c95-80a5-c82ac544c08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:05.733227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b080f6bd-50a0-4bdb-a0d2-27d7903112c3 None None] MFA Rules not processed for user `88ecd4c645174d28930a44b93cfee9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:05.740636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b080f6bd-50a0-4bdb-a0d2-27d7903112c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:05.790653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e73cab1a-5f9b-4c95-80a5-c82ac544c08f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:05.793002 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e73cab1a-5f9b-4c95-80a5-c82ac544c08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:05.895820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:05.935320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 824/1669] 10.222.0.20 () {58 vars in 1058 bytes} [Fri Jul 22 11:14:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:05.936335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080f6bd-50a0-4bdb-a0d2-27d7903112c3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:05.948186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 846/1670] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:05.956592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f925fd-87bc-44b2-99c8-077252663a9d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:05.956885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f925fd-87bc-44b2-99c8-077252663a9d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:05.957168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f925fd-87bc-44b2-99c8-077252663a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:05.962525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080f6bd-50a0-4bdb-a0d2-27d7903112c3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:05.978758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7094f0-4f46-4f79-b116-daf88f05e3e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:05.979380 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7094f0-4f46-4f79-b116-daf88f05e3e1 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:05.979632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7094f0-4f46-4f79-b116-daf88f05e3e1 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:05.979911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7094f0-4f46-4f79-b116-daf88f05e3e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:05.993694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e2f925fd-87bc-44b2-99c8-077252663a9d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:05.994905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e2f925fd-87bc-44b2-99c8-077252663a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:05.995971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bb7094f0-4f46-4f79-b116-daf88f05e3e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:06.021843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 847/1671] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:06.022893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7094f0-4f46-4f79-b116-daf88f05e3e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:06.033390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7094f0-4f46-4f79-b116-daf88f05e3e1 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:06.033555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f925fd-87bc-44b2-99c8-077252663a9d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:06.059614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f925fd-87bc-44b2-99c8-077252663a9d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:06.071658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c785b5-ffe8-4a82-8c50-71539519393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:06.072382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c785b5-ffe8-4a82-8c50-71539519393a admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:06.072670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c785b5-ffe8-4a82-8c50-71539519393a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:06.072949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c785b5-ffe8-4a82-8c50-71539519393a admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:06.073833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c785b5-ffe8-4a82-8c50-71539519393a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:06.074152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c785b5-ffe8-4a82-8c50-71539519393a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:06.078648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 825/1672] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:06.078934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c785b5-ffe8-4a82-8c50-71539519393a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:06.116856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 848/1673] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:14:06.122094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7094f0-4f46-4f79-b116-daf88f05e3e1 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:06.143838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7094f0-4f46-4f79-b116-daf88f05e3e1 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:06.164094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdfef2-5553-4f66-964d-8aafd04d67e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:06.164787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdfef2-5553-4f66-964d-8aafd04d67e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:06.165103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdfef2-5553-4f66-964d-8aafd04d67e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:06.165344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdfef2-5553-4f66-964d-8aafd04d67e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:06.166030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdfef2-5553-4f66-964d-8aafd04d67e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:06.167992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdfef2-5553-4f66-964d-8aafd04d67e2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:06.277085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 826/1674] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:06 2022] POST /identity/v3/projects => generated 409 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:06.278113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c785b5-ffe8-4a82-8c50-71539519393a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:06.309725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c785b5-ffe8-4a82-8c50-71539519393a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:06.623835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12990da3-0d0b-4565-89b2-7e1ea3561fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:06.625611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12990da3-0d0b-4565-89b2-7e1ea3561fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:06.631157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12990da3-0d0b-4565-89b2-7e1ea3561fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:06.638185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12990da3-0d0b-4565-89b2-7e1ea3561fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:06.639139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12990da3-0d0b-4565-89b2-7e1ea3561fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:06.642428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12990da3-0d0b-4565-89b2-7e1ea3561fd6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:06.739238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 849/1675] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:06 2022] POST /identity/v3/users => generated 305 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:06.740172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdfef2-5553-4f66-964d-8aafd04d67e2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:06.775499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdfef2-5553-4f66-964d-8aafd04d67e2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:06.792205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d3efd-7e0b-4f06-85de-5811a4c1e495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:06.792919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d3efd-7e0b-4f06-85de-5811a4c1e495 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:06.793214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d3efd-7e0b-4f06-85de-5811a4c1e495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:06.801415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d3efd-7e0b-4f06-85de-5811a4c1e495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:06.802537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d3efd-7e0b-4f06-85de-5811a4c1e495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:06.818447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d3efd-7e0b-4f06-85de-5811a4c1e495 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:06.869922 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 827/1676] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:06.874294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12990da3-0d0b-4565-89b2-7e1ea3561fd6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:06.890168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12990da3-0d0b-4565-89b2-7e1ea3561fd6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:06.899954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81620-711d-4e2e-9325-37b37092f12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:06.907164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81620-711d-4e2e-9325-37b37092f12d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:06.907521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81620-711d-4e2e-9325-37b37092f12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:06.908004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81620-711d-4e2e-9325-37b37092f12d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:06.908866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81620-711d-4e2e-9325-37b37092f12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:06.918202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81620-711d-4e2e-9325-37b37092f12d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:06.961093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 850/1677] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:06.962559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d3efd-7e0b-4f06-85de-5811a4c1e495 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:06.985585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d3efd-7e0b-4f06-85de-5811a4c1e495 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:07.008202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35efb47-cfed-4d5d-a113-f999a1dc3d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:07.010506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35efb47-cfed-4d5d-a113-f999a1dc3d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:07.012954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35efb47-cfed-4d5d-a113-f999a1dc3d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:07.015379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35efb47-cfed-4d5d-a113-f999a1dc3d42 admin admin] PATH_INFO: `/v3/projects/d2a9fe85fcb141b29690b3f3364eebe9/users/732508639c3943598b8a5718939a2b3f/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:07.048074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35efb47-cfed-4d5d-a113-f999a1dc3d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a9fe85fcb141b29690b3f3364eebe9, user_id=732508639c3943598b8a5718939a2b3f, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:07.054590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35efb47-cfed-4d5d-a113-f999a1dc3d42 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:07.114573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 828/1678] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:06 2022] PUT /identity/v3/projects/d2a9fe85fcb141b29690b3f3364eebe9/users/732508639c3943598b8a5718939a2b3f/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:07.126296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee42fd7-1cd7-4638-97bf-fcb85ec1a6d9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:07.126644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee42fd7-1cd7-4638-97bf-fcb85ec1a6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:07.126938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee42fd7-1cd7-4638-97bf-fcb85ec1a6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:07.195250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8ee42fd7-1cd7-4638-97bf-fcb85ec1a6d9 None None] MFA Rules not processed for user `732508639c3943598b8a5718939a2b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:07.197013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8ee42fd7-1cd7-4638-97bf-fcb85ec1a6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:07.211182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:07.351904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 851/1679] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:07.356196 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35efb47-cfed-4d5d-a113-f999a1dc3d42 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:07.582771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d2053-e842-4766-afa4-9dc0b430942d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:07.583523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d2053-e842-4766-afa4-9dc0b430942d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:07.584167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d2053-e842-4766-afa4-9dc0b430942d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:07.689268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bf3d2053-e842-4766-afa4-9dc0b430942d None None] MFA Rules not processed for user `7c2c0270618240de8fd9290d01a7413a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:07.698832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bf3d2053-e842-4766-afa4-9dc0b430942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:07.756989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35efb47-cfed-4d5d-a113-f999a1dc3d42 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:07.782932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686544e-551e-4a85-833c-8820f1f80d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:07.783928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686544e-551e-4a85-833c-8820f1f80d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:07.785784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686544e-551e-4a85-833c-8820f1f80d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:07.786187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686544e-551e-4a85-833c-8820f1f80d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:07.787044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686544e-551e-4a85-833c-8820f1f80d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:07.790524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686544e-551e-4a85-833c-8820f1f80d95 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:07.929209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 829/1680] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:07 2022] POST /identity/v3/projects => generated 409 bytes in 573 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:07.933023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686544e-551e-4a85-833c-8820f1f80d95 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:07.966779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 852/1681] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:07.978972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab414bd-1146-49f8-9920-35d3da4bc913 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:07.979528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab414bd-1146-49f8-9920-35d3da4bc913 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:07.979528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab414bd-1146-49f8-9920-35d3da4bc913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:08.091751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6ab414bd-1146-49f8-9920-35d3da4bc913 None None] MFA Rules not processed for user `7c2c0270618240de8fd9290d01a7413a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:08.093492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6ab414bd-1146-49f8-9920-35d3da4bc913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:08.177806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686544e-551e-4a85-833c-8820f1f80d95 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:08.185525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c360ea-c777-4b11-9b0e-fd9951732afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:08.187521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c360ea-c777-4b11-9b0e-fd9951732afa admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:08.187923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c360ea-c777-4b11-9b0e-fd9951732afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:08.188299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c360ea-c777-4b11-9b0e-fd9951732afa admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:08.189397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c360ea-c777-4b11-9b0e-fd9951732afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:08.197237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c360ea-c777-4b11-9b0e-fd9951732afa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:08.232452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:08.250249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 830/1682] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:07 2022] POST /identity/v3/users => generated 311 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:08.251635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c360ea-c777-4b11-9b0e-fd9951732afa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:08.298869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c360ea-c777-4b11-9b0e-fd9951732afa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:08.304004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 853/1683] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:08.316520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab414bd-1146-49f8-9920-35d3da4bc913 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:08.320499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6ab414bd-1146-49f8-9920-35d3da4bc913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:08.322192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e349e7-1f4f-49f3-997b-c9e1ca01bee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:08.326746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e349e7-1f4f-49f3-997b-c9e1ca01bee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:08.327350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e349e7-1f4f-49f3-997b-c9e1ca01bee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:08.327465 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e349e7-1f4f-49f3-997b-c9e1ca01bee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:08.357438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e349e7-1f4f-49f3-997b-c9e1ca01bee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:08.367798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e349e7-1f4f-49f3-997b-c9e1ca01bee1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:08.394353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 831/1684] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:08.396237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e349e7-1f4f-49f3-997b-c9e1ca01bee1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:08.422925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e349e7-1f4f-49f3-997b-c9e1ca01bee1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:08.432339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:08.435772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79364fb-3b7f-4a6b-8aaf-d5e9d0205db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:08.436992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79364fb-3b7f-4a6b-8aaf-d5e9d0205db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:08.437178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79364fb-3b7f-4a6b-8aaf-d5e9d0205db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:08.437500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79364fb-3b7f-4a6b-8aaf-d5e9d0205db1 admin admin] PATH_INFO: `/v3/projects/44b46d0b5e59422faa9556cef5f51cd7/users/eac9119c2f7b4ad59dc54014d9320a48/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:08.473334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab414bd-1146-49f8-9920-35d3da4bc913 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:08.481730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e1764-8843-4a6a-b012-788e77147403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:08.483346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e1764-8843-4a6a-b012-788e77147403 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:08.484589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e1764-8843-4a6a-b012-788e77147403 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:08.486009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e1764-8843-4a6a-b012-788e77147403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:08.487788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79364fb-3b7f-4a6b-8aaf-d5e9d0205db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b46d0b5e59422faa9556cef5f51cd7, user_id=eac9119c2f7b4ad59dc54014d9320a48, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:08.492863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79364fb-3b7f-4a6b-8aaf-d5e9d0205db1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:08.512863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e1764-8843-4a6a-b012-788e77147403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:08.520221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e1764-8843-4a6a-b012-788e77147403 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:08.526538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 832/1685] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:08 2022] PUT /identity/v3/projects/44b46d0b5e59422faa9556cef5f51cd7/users/eac9119c2f7b4ad59dc54014d9320a48/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:08.530732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79364fb-3b7f-4a6b-8aaf-d5e9d0205db1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:08.563338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 854/1686] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:08.778525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79364fb-3b7f-4a6b-8aaf-d5e9d0205db1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:08.793009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f7b0-59eb-4820-8b7a-e5c2b8e8d2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:08.793694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f7b0-59eb-4820-8b7a-e5c2b8e8d2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:08.793984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f7b0-59eb-4820-8b7a-e5c2b8e8d2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:08.794263 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f7b0-59eb-4820-8b7a-e5c2b8e8d2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:08.796842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f7b0-59eb-4820-8b7a-e5c2b8e8d2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:08.800293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f7b0-59eb-4820-8b7a-e5c2b8e8d2bb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:08.831283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 833/1687] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:08.834796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e1764-8843-4a6a-b012-788e77147403 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:08.865015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e1764-8843-4a6a-b012-788e77147403 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:08.888976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23053227-b3e0-46a5-b9df-73983d475330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:08.897430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23053227-b3e0-46a5-b9df-73983d475330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:08.897757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23053227-b3e0-46a5-b9df-73983d475330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:08.898012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23053227-b3e0-46a5-b9df-73983d475330 admin admin] PATH_INFO: `/v3/projects/44b46d0b5e59422faa9556cef5f51cd7/users/eac9119c2f7b4ad59dc54014d9320a48/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:08.906103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23053227-b3e0-46a5-b9df-73983d475330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b46d0b5e59422faa9556cef5f51cd7, user_id=eac9119c2f7b4ad59dc54014d9320a48, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:08.909372 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23053227-b3e0-46a5-b9df-73983d475330 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:08.987492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 855/1688] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:08 2022] PUT /identity/v3/projects/44b46d0b5e59422faa9556cef5f51cd7/users/eac9119c2f7b4ad59dc54014d9320a48/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:08.989515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1651575-c9fd-4623-9d3c-188cec4bf4e4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:08.990165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1651575-c9fd-4623-9d3c-188cec4bf4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:08.990638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1651575-c9fd-4623-9d3c-188cec4bf4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:09.076448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a1651575-c9fd-4623-9d3c-188cec4bf4e4 None None] MFA Rules not processed for user `eac9119c2f7b4ad59dc54014d9320a48`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:09.079200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1651575-c9fd-4623-9d3c-188cec4bf4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:09.306710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 834/1689] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:08 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:09.356760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23053227-b3e0-46a5-b9df-73983d475330 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:09.359167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-23053227-b3e0-46a5-b9df-73983d475330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:09.577615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23053227-b3e0-46a5-b9df-73983d475330 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:09.611173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00199cc7-152e-4ead-b4a9-4a9f967fbbf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:09.613557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00199cc7-152e-4ead-b4a9-4a9f967fbbf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:09.613686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00199cc7-152e-4ead-b4a9-4a9f967fbbf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:09.613935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00199cc7-152e-4ead-b4a9-4a9f967fbbf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:09.628463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00199cc7-152e-4ead-b4a9-4a9f967fbbf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:09.631743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00199cc7-152e-4ead-b4a9-4a9f967fbbf4 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:09.702545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 856/1690] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:09 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:10.075011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1651575-c9fd-4623-9d3c-188cec4bf4e4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:10.076794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1651575-c9fd-4623-9d3c-188cec4bf4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:10.113610 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1651575-c9fd-4623-9d3c-188cec4bf4e4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:10.140484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac9805-f571-4b16-aedb-d31ab2e05825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:10.141083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9805-f571-4b16-aedb-d31ab2e05825 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:10.141374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9805-f571-4b16-aedb-d31ab2e05825 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:10.141655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9805-f571-4b16-aedb-d31ab2e05825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:10.219740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4f253-bf72-4c21-9b25-16c32fe20f29 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:10.220055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4f253-bf72-4c21-9b25-16c32fe20f29 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:10.220319 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4f253-bf72-4c21-9b25-16c32fe20f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:10.254200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:10.266164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bdf4f253-bf72-4c21-9b25-16c32fe20f29 None None] MFA Rules not processed for user `7c2c0270618240de8fd9290d01a7413a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:10.270562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bdf4f253-bf72-4c21-9b25-16c32fe20f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:10.301682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac9805-f571-4b16-aedb-d31ab2e05825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:10.309831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac9805-f571-4b16-aedb-d31ab2e05825 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:10.337897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 835/1691] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:10.405265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 857/1692] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:10.570110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744aae1-1ed5-474c-b49d-ed5fdce215c2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:10.570521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744aae1-1ed5-474c-b49d-ed5fdce215c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:10.570905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744aae1-1ed5-474c-b49d-ed5fdce215c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:10.607359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7744aae1-1ed5-474c-b49d-ed5fdce215c2 None None] MFA Rules not processed for user `691877dce1024aeda61629073d74602b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:10.608773 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7744aae1-1ed5-474c-b49d-ed5fdce215c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:10.609826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e8cd-8573-404c-83a2-3f27f5c83dff None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:10.613277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e8cd-8573-404c-83a2-3f27f5c83dff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:10.613935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e8cd-8573-404c-83a2-3f27f5c83dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:10.671203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b1a4e8cd-8573-404c-83a2-3f27f5c83dff None None] MFA Rules not processed for user `da75b78b267f47e8aaaafb41e3acb61a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:10.674837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b1a4e8cd-8573-404c-83a2-3f27f5c83dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:10.808957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 836/1693] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:10.884177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:10.984164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 858/1694] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:11.930527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744aae1-1ed5-474c-b49d-ed5fdce215c2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:12.083432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744aae1-1ed5-474c-b49d-ed5fdce215c2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:12.090849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ec795-d1d7-4c40-8d63-2117d702aab2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:12.094302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ec795-d1d7-4c40-8d63-2117d702aab2 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:12.094845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ec795-d1d7-4c40-8d63-2117d702aab2 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:12.095367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ec795-d1d7-4c40-8d63-2117d702aab2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:12.123519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ec795-d1d7-4c40-8d63-2117d702aab2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:12.126018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ec795-d1d7-4c40-8d63-2117d702aab2 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:12.165529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 837/1695] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:12.230333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801fa8d-bbb9-4ed0-a275-34533b037ca2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:12.230936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801fa8d-bbb9-4ed0-a275-34533b037ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:12.231018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801fa8d-bbb9-4ed0-a275-34533b037ca2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:12.234222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 859/1696] 10.222.0.20 () {54 vars in 912 bytes} [Fri Jul 22 11:14:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:14:12.267364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f05c8d-d28a-41b8-95ef-435f481e0c94 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:12.271037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f05c8d-d28a-41b8-95ef-435f481e0c94 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:12.272888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f05c8d-d28a-41b8-95ef-435f481e0c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:12.369897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-69f05c8d-d28a-41b8-95ef-435f481e0c94 None None] MFA Rules not processed for user `0a0ce1804b514b42845ab33c7bfee309`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:12.372011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-69f05c8d-d28a-41b8-95ef-435f481e0c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:12.594598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 838/1697] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:14:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:12.699297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801fa8d-bbb9-4ed0-a275-34533b037ca2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:12.701447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c801fa8d-bbb9-4ed0-a275-34533b037ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:12.824209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801fa8d-bbb9-4ed0-a275-34533b037ca2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:12.868146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f90a4-7bbe-455b-9f0f-28b3d57bb8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:12.868850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f90a4-7bbe-455b-9f0f-28b3d57bb8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:12.869145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f90a4-7bbe-455b-9f0f-28b3d57bb8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:12.869430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f90a4-7bbe-455b-9f0f-28b3d57bb8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:12.983162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f90a4-7bbe-455b-9f0f-28b3d57bb8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:12.984983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f90a4-7bbe-455b-9f0f-28b3d57bb8e4 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:13.038932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 860/1698] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:16.101899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254a7cd-7f6a-4940-a338-f16586e1ca64 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:16.102578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254a7cd-7f6a-4940-a338-f16586e1ca64 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:16.102877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254a7cd-7f6a-4940-a338-f16586e1ca64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:16.133592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5254a7cd-7f6a-4940-a338-f16586e1ca64 None None] MFA Rules not processed for user `da75b78b267f47e8aaaafb41e3acb61a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:16.135469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5254a7cd-7f6a-4940-a338-f16586e1ca64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:16.176756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 839/1699] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:16.260087 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbb5b4-e97b-4e20-b6fe-016df12f8c22 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:16.261397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbb5b4-e97b-4e20-b6fe-016df12f8c22 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:16.261951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbb5b4-e97b-4e20-b6fe-016df12f8c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:16.330039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-50dbb5b4-e97b-4e20-b6fe-016df12f8c22 None None] MFA Rules not processed for user `0a0ce1804b514b42845ab33c7bfee309`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:16.332603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-50dbb5b4-e97b-4e20-b6fe-016df12f8c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:16.359552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 861/1700] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jul 22 11:14:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:16.368338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303692d-2b87-4c26-a329-f1973396ca0a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:16.368842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303692d-2b87-4c26-a329-f1973396ca0a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:16.369322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303692d-2b87-4c26-a329-f1973396ca0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:16.384048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbb5b4-e97b-4e20-b6fe-016df12f8c22 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:16.385772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-50dbb5b4-e97b-4e20-b6fe-016df12f8c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:16.407525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbb5b4-e97b-4e20-b6fe-016df12f8c22 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:16.420104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6dbdf-dc19-4cf0-9309-e4f2c27896dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:16.420804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6dbdf-dc19-4cf0-9309-e4f2c27896dc service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:16.421098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6dbdf-dc19-4cf0-9309-e4f2c27896dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:16.421374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6dbdf-dc19-4cf0-9309-e4f2c27896dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:16.433044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6dbdf-dc19-4cf0-9309-e4f2c27896dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:16.434739 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6dbdf-dc19-4cf0-9309-e4f2c27896dc service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:16.462442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 862/1701] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:16.473428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:16.559953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4303692d-2b87-4c26-a329-f1973396ca0a None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:16.561846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4303692d-2b87-4c26-a329-f1973396ca0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:16.776055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 840/1702] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:14:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:17.114447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6dbdf-dc19-4cf0-9309-e4f2c27896dc service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:17.117314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e3b6dbdf-dc19-4cf0-9309-e4f2c27896dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:17.488734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6dbdf-dc19-4cf0-9309-e4f2c27896dc service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:17.511188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05a5e7-e7ab-456c-8057-ad778ca91b79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:17.511910 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05a5e7-e7ab-456c-8057-ad778ca91b79 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:17.512197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05a5e7-e7ab-456c-8057-ad778ca91b79 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:17.512645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05a5e7-e7ab-456c-8057-ad778ca91b79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:17.546167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05a5e7-e7ab-456c-8057-ad778ca91b79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:17.552550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05a5e7-e7ab-456c-8057-ad778ca91b79 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:17.596638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 863/1703] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:14:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:14:19.114278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba932fd-d8fe-4823-8f94-f931ba2f0a45 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:19.115788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba932fd-d8fe-4823-8f94-f931ba2f0a45 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:19.117280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba932fd-d8fe-4823-8f94-f931ba2f0a45 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:19.125363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 841/1704] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:14:19 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:14:19.144600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690d70d-f526-4a7f-8407-ff181d221d8e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:19.145438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690d70d-f526-4a7f-8407-ff181d221d8e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:19.145954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690d70d-f526-4a7f-8407-ff181d221d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:19.221569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9690d70d-f526-4a7f-8407-ff181d221d8e None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:19.224310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9690d70d-f526-4a7f-8407-ff181d221d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:19.260169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 864/1705] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:14:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:19.318480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba932fd-d8fe-4823-8f94-f931ba2f0a45 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:19.320266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2ba932fd-d8fe-4823-8f94-f931ba2f0a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:19.333560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690d70d-f526-4a7f-8407-ff181d221d8e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:19.335972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9690d70d-f526-4a7f-8407-ff181d221d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:19.360651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690d70d-f526-4a7f-8407-ff181d221d8e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:19.381308 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c96cea-e505-4fb7-a8eb-4626fa3e2965 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:19.385205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c96cea-e505-4fb7-a8eb-4626fa3e2965 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:19.386276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c96cea-e505-4fb7-a8eb-4626fa3e2965 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:19.386454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c96cea-e505-4fb7-a8eb-4626fa3e2965 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:19.454215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c96cea-e505-4fb7-a8eb-4626fa3e2965 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:19.465561 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c96cea-e505-4fb7-a8eb-4626fa3e2965 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:19.551423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 865/1706] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:19.622285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba932fd-d8fe-4823-8f94-f931ba2f0a45 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:19.684378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:19.703049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:19.703435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:19.703650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] PATH_INFO: `/v3/users/81cf75f532a44b27930a8da72700d21a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:19.719018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] RBAC: Authorizing `identity:delete_user(user_id=81cf75f532a44b27930a8da72700d21a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:19.742983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:20.038251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81cf75f532a44b27930a8da72700d21a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:20.052272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81cf75f532a44b27930a8da72700d21a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:20.078604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81cf75f532a44b27930a8da72700d21a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:20.116081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81cf75f532a44b27930a8da72700d21a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:20.117250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81cf75f532a44b27930a8da72700d21a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:20.143526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 842/1707] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:19 2022] DELETE /identity/v3/users/81cf75f532a44b27930a8da72700d21a => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:20.150697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7150487-a03a-4433-8af4-fe9d3578f3e3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:20.151249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7150487-a03a-4433-8af4-fe9d3578f3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:20.151249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7150487-a03a-4433-8af4-fe9d3578f3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:20.234575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e7150487-a03a-4433-8af4-fe9d3578f3e3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:20.245474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e7150487-a03a-4433-8af4-fe9d3578f3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:20.455508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 866/1708] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:20.470516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:20.480773 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:20.496946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:20.626626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5edfb4-6ccf-4f7d-9d3f-42070b2716da admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:20.655222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359265f-64be-4e97-8719-70a9b5ab657f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:20.656367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359265f-64be-4e97-8719-70a9b5ab657f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:20.656947 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359265f-64be-4e97-8719-70a9b5ab657f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:20.657461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359265f-64be-4e97-8719-70a9b5ab657f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:20.689846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359265f-64be-4e97-8719-70a9b5ab657f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:20.693280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359265f-64be-4e97-8719-70a9b5ab657f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:20.727742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 843/1709] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:21.288382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7150487-a03a-4433-8af4-fe9d3578f3e3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:21.289999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e7150487-a03a-4433-8af4-fe9d3578f3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:21.336175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7150487-a03a-4433-8af4-fe9d3578f3e3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:21.357927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:21.364513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:21.364596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:21.364851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] PATH_INFO: `/v3/projects/a0ca7d833cb44d02aabc2d345df833f3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:21.371085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ca7d833cb44d02aabc2d345df833f3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:21.378536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:21.432249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:21.580149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ca7d833cb44d02aabc2d345df833f3', 'request_id': 'req-6d7965e8-e203-404a-b471-ec96194a0844'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:21.680223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ca7d833cb44d02aabc2d345df833f3', 'request_id': 'req-6d7965e8-e203-404a-b471-ec96194a0844'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:21.737304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ca7d833cb44d02aabc2d345df833f3', 'request_id': 'req-6d7965e8-e203-404a-b471-ec96194a0844'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:21.836806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] The token cache is being invalidate because project a0ca7d833cb44d02aabc2d345df833f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:21.843841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:21.845377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6d7965e8-e203-404a-b471-ec96194a0844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:21.853219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 867/1710] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:21 2022] DELETE /identity/v3/projects/a0ca7d833cb44d02aabc2d345df833f3 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:21.859148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b7006-75dc-454f-a747-32d33af1c685 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:21.860254 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b7006-75dc-454f-a747-32d33af1c685 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:21.860719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b7006-75dc-454f-a747-32d33af1c685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:21.936651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-472b7006-75dc-454f-a747-32d33af1c685 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:21.938366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-472b7006-75dc-454f-a747-32d33af1c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:22.264800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 844/1711] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:22.273433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0fb0b-06e0-4dbc-b1a7-f0c375cac669 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:22.279064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0fb0b-06e0-4dbc-b1a7-f0c375cac669 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:22.280746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0fb0b-06e0-4dbc-b1a7-f0c375cac669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:22.375776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1fa0fb0b-06e0-4dbc-b1a7-f0c375cac669 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:22.385015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1fa0fb0b-06e0-4dbc-b1a7-f0c375cac669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:22.440900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 868/1712] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:22.446213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b7006-75dc-454f-a747-32d33af1c685 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:22.469348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b7006-75dc-454f-a747-32d33af1c685 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:22.485396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189cc89-0009-4b36-8c2d-1a1c1bf2e9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:22.488715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189cc89-0009-4b36-8c2d-1a1c1bf2e9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:22.493443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189cc89-0009-4b36-8c2d-1a1c1bf2e9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:22.496006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189cc89-0009-4b36-8c2d-1a1c1bf2e9c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:22.499081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189cc89-0009-4b36-8c2d-1a1c1bf2e9c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:22.499829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189cc89-0009-4b36-8c2d-1a1c1bf2e9c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:22.506954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189cc89-0009-4b36-8c2d-1a1c1bf2e9c2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:22.546645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 845/1713] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:22.551081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0fb0b-06e0-4dbc-b1a7-f0c375cac669 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:22.599469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0fb0b-06e0-4dbc-b1a7-f0c375cac669 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:22.639212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d743094-5195-44d7-b039-69e1616d9fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:22.643839 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743094-5195-44d7-b039-69e1616d9fbd admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:22.645268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743094-5195-44d7-b039-69e1616d9fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:22.646275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743094-5195-44d7-b039-69e1616d9fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:22.650344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d743094-5195-44d7-b039-69e1616d9fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:22.668032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d743094-5195-44d7-b039-69e1616d9fbd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:22.765948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 869/1714] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:22 2022] POST /identity/v3/projects => generated 381 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:22.771837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189cc89-0009-4b36-8c2d-1a1c1bf2e9c2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:22.805693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189cc89-0009-4b36-8c2d-1a1c1bf2e9c2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:23.129474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f5b7a-182a-4787-8639-fc2bda1975d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:23.137316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f5b7a-182a-4787-8639-fc2bda1975d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:23.137316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f5b7a-182a-4787-8639-fc2bda1975d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:23.137710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f5b7a-182a-4787-8639-fc2bda1975d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:23.138325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f5b7a-182a-4787-8639-fc2bda1975d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:23.142155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f5b7a-182a-4787-8639-fc2bda1975d3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:23.223219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 846/1715] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:22 2022] POST /identity/v3/users => generated 291 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:23.229110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d743094-5195-44d7-b039-69e1616d9fbd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:23.244341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d743094-5195-44d7-b039-69e1616d9fbd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:23.269907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b94c-4a79-4b03-9dda-a8ab455a72b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:23.270798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b94c-4a79-4b03-9dda-a8ab455a72b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:23.271090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b94c-4a79-4b03-9dda-a8ab455a72b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:23.271403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b94c-4a79-4b03-9dda-a8ab455a72b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:23.280102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269b94c-4a79-4b03-9dda-a8ab455a72b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:23.281990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269b94c-4a79-4b03-9dda-a8ab455a72b3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:23.341133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 870/1716] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:23.345358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f5b7a-182a-4787-8639-fc2bda1975d3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:23.360930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f5b7a-182a-4787-8639-fc2bda1975d3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:23.373214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37ce9c-cf7f-4e9a-8905-47ceb08c6592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:23.375952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37ce9c-cf7f-4e9a-8905-47ceb08c6592 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:23.376320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37ce9c-cf7f-4e9a-8905-47ceb08c6592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:23.377810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37ce9c-cf7f-4e9a-8905-47ceb08c6592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:23.380897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37ce9c-cf7f-4e9a-8905-47ceb08c6592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:23.383951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37ce9c-cf7f-4e9a-8905-47ceb08c6592 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:23.418517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b94c-4a79-4b03-9dda-a8ab455a72b3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:23.420176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 847/1717] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:23.431810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b94c-4a79-4b03-9dda-a8ab455a72b3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:23.449310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2eea24-25b9-477d-8679-958271312171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:23.450607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2eea24-25b9-477d-8679-958271312171 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:23.451072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2eea24-25b9-477d-8679-958271312171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:23.451692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2eea24-25b9-477d-8679-958271312171 admin admin] PATH_INFO: `/v3/projects/a3f3d6309ebc43ba8b8d294b2615e564/users/a06f477c35004033a3d13511e5eb612c/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:23.486684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2eea24-25b9-477d-8679-958271312171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f3d6309ebc43ba8b8d294b2615e564, user_id=a06f477c35004033a3d13511e5eb612c, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:23.490250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2eea24-25b9-477d-8679-958271312171 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:23.542630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f862ab4-7822-4ca9-98a5-ae88786d03dd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:23.546223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f862ab4-7822-4ca9-98a5-ae88786d03dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:23.546704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f862ab4-7822-4ca9-98a5-ae88786d03dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:23.548148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 871/1718] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:23 2022] PUT /identity/v3/projects/a3f3d6309ebc43ba8b8d294b2615e564/users/a06f477c35004033a3d13511e5eb612c/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:23.605637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7f862ab4-7822-4ca9-98a5-ae88786d03dd None None] MFA Rules not processed for user `a06f477c35004033a3d13511e5eb612c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:23.616235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7f862ab4-7822-4ca9-98a5-ae88786d03dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:23.627389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:23.832092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 848/1719] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:23.834994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2eea24-25b9-477d-8679-958271312171 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:24.021388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2eea24-25b9-477d-8679-958271312171 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:24.027681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd669b09-d357-4a50-87c1-a70f7ea22fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:24.029377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd669b09-d357-4a50-87c1-a70f7ea22fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:24.029897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd669b09-d357-4a50-87c1-a70f7ea22fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:24.030220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd669b09-d357-4a50-87c1-a70f7ea22fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:24.031015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd669b09-d357-4a50-87c1-a70f7ea22fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:24.033982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd669b09-d357-4a50-87c1-a70f7ea22fd7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:24.089141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 872/1720] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:23 2022] POST /identity/v3/projects => generated 381 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:24.090714 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f862ab4-7822-4ca9-98a5-ae88786d03dd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:24.119695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f862ab4-7822-4ca9-98a5-ae88786d03dd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:24.229630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934127c9-d19f-40bb-b191-30a0d293fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:24.231436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934127c9-d19f-40bb-b191-30a0d293fe11 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:24.232746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934127c9-d19f-40bb-b191-30a0d293fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:24.232746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934127c9-d19f-40bb-b191-30a0d293fe11 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:24.232995 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934127c9-d19f-40bb-b191-30a0d293fe11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:24.235186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934127c9-d19f-40bb-b191-30a0d293fe11 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:24.260875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 849/1721] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:24 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:24.264685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd669b09-d357-4a50-87c1-a70f7ea22fd7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:24.278269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd669b09-d357-4a50-87c1-a70f7ea22fd7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:24.282661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b05070-ece8-4f43-84c8-169ec71094ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:24.284041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b05070-ece8-4f43-84c8-169ec71094ae admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:24.284414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b05070-ece8-4f43-84c8-169ec71094ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:24.284762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b05070-ece8-4f43-84c8-169ec71094ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:24.285613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b05070-ece8-4f43-84c8-169ec71094ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:24.288282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b05070-ece8-4f43-84c8-169ec71094ae admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:24.305818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 873/1722] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:24.311309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934127c9-d19f-40bb-b191-30a0d293fe11 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:24.330643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934127c9-d19f-40bb-b191-30a0d293fe11 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:24.340823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268de003-ffbf-4927-b259-a3c8923208a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:24.342056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268de003-ffbf-4927-b259-a3c8923208a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:24.342631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268de003-ffbf-4927-b259-a3c8923208a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:24.343249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268de003-ffbf-4927-b259-a3c8923208a6 admin admin] PATH_INFO: `/v3/projects/e74e1943470a4baaa1b0a00d95432073/users/65a7d27f33a54080880ebc724dabdf59/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:24.364507 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268de003-ffbf-4927-b259-a3c8923208a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74e1943470a4baaa1b0a00d95432073, user_id=65a7d27f33a54080880ebc724dabdf59, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:24.367228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268de003-ffbf-4927-b259-a3c8923208a6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:24.403206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 850/1723] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:24 2022] PUT /identity/v3/projects/e74e1943470a4baaa1b0a00d95432073/users/65a7d27f33a54080880ebc724dabdf59/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:24.408126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b05070-ece8-4f43-84c8-169ec71094ae admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:24.425615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b05070-ece8-4f43-84c8-169ec71094ae admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:24.436657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:24.536012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d83627-f870-4fab-be1d-dcbda034e2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:24.537847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d83627-f870-4fab-be1d-dcbda034e2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:24.538296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d83627-f870-4fab-be1d-dcbda034e2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:24.538719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d83627-f870-4fab-be1d-dcbda034e2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:24.539657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d83627-f870-4fab-be1d-dcbda034e2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:24.541514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d83627-f870-4fab-be1d-dcbda034e2a2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:24.559788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 874/1724] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:24.560163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268de003-ffbf-4927-b259-a3c8923208a6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:24.581964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268de003-ffbf-4927-b259-a3c8923208a6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:24.589437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb89b-6aac-4dff-8fdf-e15ff0f9913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:24.590389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb89b-6aac-4dff-8fdf-e15ff0f9913e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:24.590676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb89b-6aac-4dff-8fdf-e15ff0f9913e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:24.591642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb89b-6aac-4dff-8fdf-e15ff0f9913e admin admin] PATH_INFO: `/v3/projects/e74e1943470a4baaa1b0a00d95432073/users/65a7d27f33a54080880ebc724dabdf59/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:24.593729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cb89b-6aac-4dff-8fdf-e15ff0f9913e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74e1943470a4baaa1b0a00d95432073, user_id=65a7d27f33a54080880ebc724dabdf59, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:24.596778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cb89b-6aac-4dff-8fdf-e15ff0f9913e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:24.612360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 851/1725] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:24 2022] PUT /identity/v3/projects/e74e1943470a4baaa1b0a00d95432073/users/65a7d27f33a54080880ebc724dabdf59/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:24.623634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee825ad-1884-4687-827e-df0ecc4af248 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:24.624010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee825ad-1884-4687-827e-df0ecc4af248 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:24.624351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee825ad-1884-4687-827e-df0ecc4af248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:24.701240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7ee825ad-1884-4687-827e-df0ecc4af248 None None] MFA Rules not processed for user `65a7d27f33a54080880ebc724dabdf59`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:24.702648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7ee825ad-1884-4687-827e-df0ecc4af248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:24.872761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 875/1726] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:24.881533 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb89b-6aac-4dff-8fdf-e15ff0f9913e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:24.886004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-814cb89b-6aac-4dff-8fdf-e15ff0f9913e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:25.072030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb89b-6aac-4dff-8fdf-e15ff0f9913e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:25.078442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d871f-1a91-4212-b064-26601a29e562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:25.079426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d871f-1a91-4212-b064-26601a29e562 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:25.079599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d871f-1a91-4212-b064-26601a29e562 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:25.079906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d871f-1a91-4212-b064-26601a29e562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:25.096830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d871f-1a91-4212-b064-26601a29e562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:25.098929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d871f-1a91-4212-b064-26601a29e562 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:25.112730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 852/1727] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:25.289311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee825ad-1884-4687-827e-df0ecc4af248 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:25.291800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7ee825ad-1884-4687-827e-df0ecc4af248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:25.300830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412e3d6-63d3-47f1-9208-a39dc5bc2ec7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:25.301400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412e3d6-63d3-47f1-9208-a39dc5bc2ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:25.301400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412e3d6-63d3-47f1-9208-a39dc5bc2ec7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:25.303404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 853/1728] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:14:25.313856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee825ad-1884-4687-827e-df0ecc4af248 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:25.319236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f7775-5e7c-43ad-bc94-3871e9c27521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:25.321683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f7775-5e7c-43ad-bc94-3871e9c27521 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:25.322054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f7775-5e7c-43ad-bc94-3871e9c27521 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:25.322400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f7775-5e7c-43ad-bc94-3871e9c27521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:25.334532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22d239-6f6b-418e-b9f0-8048f5ed7c88 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:25.334834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22d239-6f6b-418e-b9f0-8048f5ed7c88 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:25.335104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22d239-6f6b-418e-b9f0-8048f5ed7c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:25.347710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:25.367998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dc22d239-6f6b-418e-b9f0-8048f5ed7c88 None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:25.370246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dc22d239-6f6b-418e-b9f0-8048f5ed7c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:25.421333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f7775-5e7c-43ad-bc94-3871e9c27521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:25.422965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f7775-5e7c-43ad-bc94-3871e9c27521 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:25.443451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 876/1729] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:25.483517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 854/1730] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:14:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:25.495853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f7775-5e7c-43ad-bc94-3871e9c27521 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:25.497330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-402f7775-5e7c-43ad-bc94-3871e9c27521 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:25.518291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f7775-5e7c-43ad-bc94-3871e9c27521 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:25.524693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee66fd-962d-49ce-b742-dcc7306e8640 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:25.525375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee66fd-962d-49ce-b742-dcc7306e8640 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:25.525664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee66fd-962d-49ce-b742-dcc7306e8640 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:25.525945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee66fd-962d-49ce-b742-dcc7306e8640 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:25.538390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee66fd-962d-49ce-b742-dcc7306e8640 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:25.540764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee66fd-962d-49ce-b742-dcc7306e8640 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:25.553216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 877/1731] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:27.325160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22d239-6f6b-418e-b9f0-8048f5ed7c88 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:27.329275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dc22d239-6f6b-418e-b9f0-8048f5ed7c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:27.545950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22d239-6f6b-418e-b9f0-8048f5ed7c88 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:27.569769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:27.572038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:27.572513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:27.574908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] PATH_INFO: `/v3/users/732508639c3943598b8a5718939a2b3f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:27.577680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=732508639c3943598b8a5718939a2b3f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:27.587438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:27.767985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '732508639c3943598b8a5718939a2b3f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:27.790478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '732508639c3943598b8a5718939a2b3f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:27.811702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '732508639c3943598b8a5718939a2b3f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:27.859085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '732508639c3943598b8a5718939a2b3f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:27.862610 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '732508639c3943598b8a5718939a2b3f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:27.881767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 855/1732] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:27 2022] DELETE /identity/v3/users/732508639c3943598b8a5718939a2b3f => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:27.893871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aa6ea-99a0-4616-a360-37fd0c4659c6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:27.895627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aa6ea-99a0-4616-a360-37fd0c4659c6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:27.896290 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aa6ea-99a0-4616-a360-37fd0c4659c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:27.934508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1f0aa6ea-99a0-4616-a360-37fd0c4659c6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:27.937861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1f0aa6ea-99a0-4616-a360-37fd0c4659c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:27.971935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:28.109745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 878/1733] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:28.135957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:28.137567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:28.179058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:28.265414 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b906e-72c8-4a55-b6ef-bd9d31801613 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:28.271157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d2063-9b93-4aa0-bd8e-5d6e34e100e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:28.272182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d2063-9b93-4aa0-bd8e-5d6e34e100e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:28.272632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d2063-9b93-4aa0-bd8e-5d6e34e100e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:28.273258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d2063-9b93-4aa0-bd8e-5d6e34e100e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:28.286762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d2063-9b93-4aa0-bd8e-5d6e34e100e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:28.289070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d2063-9b93-4aa0-bd8e-5d6e34e100e0 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:28.321457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 856/1734] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:28.914319 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aa6ea-99a0-4616-a360-37fd0c4659c6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:28.916025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1f0aa6ea-99a0-4616-a360-37fd0c4659c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:28.946195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aa6ea-99a0-4616-a360-37fd0c4659c6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:28.966543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:28.969251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:28.972652 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:28.973133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] PATH_INFO: `/v3/projects/d2a9fe85fcb141b29690b3f3364eebe9` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:28.976325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a9fe85fcb141b29690b3f3364eebe9)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:28.981300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:29.152947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a9fe85fcb141b29690b3f3364eebe9', 'request_id': 'req-dd03a7a5-9ea4-4785-8143-b09474e34e92'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.220056 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a9fe85fcb141b29690b3f3364eebe9', 'request_id': 'req-dd03a7a5-9ea4-4785-8143-b09474e34e92'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.250334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a9fe85fcb141b29690b3f3364eebe9', 'request_id': 'req-dd03a7a5-9ea4-4785-8143-b09474e34e92'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.290004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] The token cache is being invalidate because project d2a9fe85fcb141b29690b3f3364eebe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:29.291574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.292016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.296536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 879/1735] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:28 2022] DELETE /identity/v3/projects/d2a9fe85fcb141b29690b3f3364eebe9 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:29.302265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d2063-9b93-4aa0-bd8e-5d6e34e100e0 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:29.317989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d2063-9b93-4aa0-bd8e-5d6e34e100e0 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:29.418377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:29.419434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:29.419744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:29.420033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] PATH_INFO: `/v3/users/eac9119c2f7b4ad59dc54014d9320a48` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:29.421302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac9119c2f7b4ad59dc54014d9320a48)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:29.424191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:29.501499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac9119c2f7b4ad59dc54014d9320a48'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.529446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac9119c2f7b4ad59dc54014d9320a48'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.546019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac9119c2f7b4ad59dc54014d9320a48'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.575983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac9119c2f7b4ad59dc54014d9320a48'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.577451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-501c7b0c-199e-44fb-a366-ea593d0e6a1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac9119c2f7b4ad59dc54014d9320a48'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:29.596299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 857/1736] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:29 2022] DELETE /identity/v3/users/eac9119c2f7b4ad59dc54014d9320a48 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:30.160035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:30.161562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:30.178795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03a7a5-9ea4-4785-8143-b09474e34e92 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:30.271347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:30.272764 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:30.273224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:30.273663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] PATH_INFO: `/v3/projects/44b46d0b5e59422faa9556cef5f51cd7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:30.275928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b46d0b5e59422faa9556cef5f51cd7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:30.279843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:30.348002 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:30.358415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b46d0b5e59422faa9556cef5f51cd7', 'request_id': 'req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:30.368278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b46d0b5e59422faa9556cef5f51cd7', 'request_id': 'req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:30.377435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b46d0b5e59422faa9556cef5f51cd7', 'request_id': 'req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:30.398655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] The token cache is being invalidate because project 44b46d0b5e59422faa9556cef5f51cd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:30.399039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:30.399884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7aef5f25-b1f0-4bb8-8ec3-e4ba5cadbf0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:30.405523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 880/1737] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:30 2022] DELETE /identity/v3/projects/44b46d0b5e59422faa9556cef5f51cd7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:30.408892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86882e2-608d-48be-87b2-9d113c0fd434 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:30.413753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86882e2-608d-48be-87b2-9d113c0fd434 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:30.414401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86882e2-608d-48be-87b2-9d113c0fd434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:30.441626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a86882e2-608d-48be-87b2-9d113c0fd434 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:30.448944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a86882e2-608d-48be-87b2-9d113c0fd434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:30.494563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:30.536122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 858/1738] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:30.546147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e779d3e-7700-4bf3-a5bf-346796c25734 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:30.546445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e779d3e-7700-4bf3-a5bf-346796c25734 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:30.546728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e779d3e-7700-4bf3-a5bf-346796c25734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:30.574393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4e779d3e-7700-4bf3-a5bf-346796c25734 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:30.575901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4e779d3e-7700-4bf3-a5bf-346796c25734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:30.587305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 881/1739] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:30.588944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86882e2-608d-48be-87b2-9d113c0fd434 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:30.603751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86882e2-608d-48be-87b2-9d113c0fd434 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:30.608859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ca60a-896e-4403-91e1-2b66b2fd1b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:30.609546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ca60a-896e-4403-91e1-2b66b2fd1b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:30.609836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ca60a-896e-4403-91e1-2b66b2fd1b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:30.610117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ca60a-896e-4403-91e1-2b66b2fd1b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:30.610899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ca60a-896e-4403-91e1-2b66b2fd1b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:30.611784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ca60a-896e-4403-91e1-2b66b2fd1b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:30.613472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ca60a-896e-4403-91e1-2b66b2fd1b1b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:30.628669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 859/1740] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:30.632395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e779d3e-7700-4bf3-a5bf-346796c25734 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:30.641955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e779d3e-7700-4bf3-a5bf-346796c25734 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:30.652872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821c446-6adc-47f3-a558-eebae82951d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:30.653718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821c446-6adc-47f3-a558-eebae82951d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:30.654126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821c446-6adc-47f3-a558-eebae82951d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:30.654538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821c446-6adc-47f3-a558-eebae82951d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:30.655377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821c446-6adc-47f3-a558-eebae82951d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:30.659662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821c446-6adc-47f3-a558-eebae82951d4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:30.699872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 882/1741] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:30 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:30.706280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ca60a-896e-4403-91e1-2b66b2fd1b1b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:30.718798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ca60a-896e-4403-91e1-2b66b2fd1b1b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:30.823542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf80b54-97b8-40ed-a210-078029d07363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:30.824154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf80b54-97b8-40ed-a210-078029d07363 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:30.824441 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf80b54-97b8-40ed-a210-078029d07363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:30.824721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf80b54-97b8-40ed-a210-078029d07363 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:30.825724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf80b54-97b8-40ed-a210-078029d07363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:30.827912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf80b54-97b8-40ed-a210-078029d07363 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:30.853492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 860/1742] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:30 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:30.860532 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821c446-6adc-47f3-a558-eebae82951d4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:30.875652 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821c446-6adc-47f3-a558-eebae82951d4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:30.891629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5310c56-fe78-4b73-99e4-7676d7cc2c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:30.893180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5310c56-fe78-4b73-99e4-7676d7cc2c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:30.893611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5310c56-fe78-4b73-99e4-7676d7cc2c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:30.893988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5310c56-fe78-4b73-99e4-7676d7cc2c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:30.895033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5310c56-fe78-4b73-99e4-7676d7cc2c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:30.901522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5310c56-fe78-4b73-99e4-7676d7cc2c47 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:30.918789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 883/1743] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:30.923244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf80b54-97b8-40ed-a210-078029d07363 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:30.939121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf80b54-97b8-40ed-a210-078029d07363 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:30.953566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058bd9f-a537-44d7-9d47-92fc1358f766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:30.956832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058bd9f-a537-44d7-9d47-92fc1358f766 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:30.956832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058bd9f-a537-44d7-9d47-92fc1358f766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:30.957125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058bd9f-a537-44d7-9d47-92fc1358f766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:30.957957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058bd9f-a537-44d7-9d47-92fc1358f766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:30.960681 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058bd9f-a537-44d7-9d47-92fc1358f766 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:30.975393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 861/1744] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:30.980353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5310c56-fe78-4b73-99e4-7676d7cc2c47 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:31.031953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5310c56-fe78-4b73-99e4-7676d7cc2c47 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:31.042969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6ab89-9e47-48d2-af3e-1f89c51bea2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:31.045973 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6ab89-9e47-48d2-af3e-1f89c51bea2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:31.046656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6ab89-9e47-48d2-af3e-1f89c51bea2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:31.047190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6ab89-9e47-48d2-af3e-1f89c51bea2b admin admin] PATH_INFO: `/v3/projects/6372c972cbde4782bc80760a9a671315/users/b6ac835d50a14333bd10f9cc6c02c937/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:31.084103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6ab89-9e47-48d2-af3e-1f89c51bea2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6372c972cbde4782bc80760a9a671315, user_id=b6ac835d50a14333bd10f9cc6c02c937, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:31.087066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6ab89-9e47-48d2-af3e-1f89c51bea2b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:31.105683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 884/1745] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:30 2022] PUT /identity/v3/projects/6372c972cbde4782bc80760a9a671315/users/b6ac835d50a14333bd10f9cc6c02c937/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:31.119226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69abeaf-ac8a-4ce4-a1c5-fa85d0780576 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:31.119556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69abeaf-ac8a-4ce4-a1c5-fa85d0780576 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:31.119846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69abeaf-ac8a-4ce4-a1c5-fa85d0780576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:31.159764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e69abeaf-ac8a-4ce4-a1c5-fa85d0780576 None None] MFA Rules not processed for user `b6ac835d50a14333bd10f9cc6c02c937`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:31.161756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e69abeaf-ac8a-4ce4-a1c5-fa85d0780576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:31.289597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 862/1746] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:31.299649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6ab89-9e47-48d2-af3e-1f89c51bea2b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:31.410442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6ab89-9e47-48d2-af3e-1f89c51bea2b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:31.418631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7148f3-2161-46d8-8f90-ec5771405d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:31.420015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7148f3-2161-46d8-8f90-ec5771405d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:31.420628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7148f3-2161-46d8-8f90-ec5771405d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:31.421206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7148f3-2161-46d8-8f90-ec5771405d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:31.422696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7148f3-2161-46d8-8f90-ec5771405d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:31.427522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7148f3-2161-46d8-8f90-ec5771405d98 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:31.478462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 885/1747] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:31 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:31.484346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69abeaf-ac8a-4ce4-a1c5-fa85d0780576 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:31.497239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69abeaf-ac8a-4ce4-a1c5-fa85d0780576 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:31.552133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:31.633047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71df0b0-82ed-498f-ad47-fd8569230ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:31.636185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71df0b0-82ed-498f-ad47-fd8569230ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:31.636511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71df0b0-82ed-498f-ad47-fd8569230ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:31.636874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71df0b0-82ed-498f-ad47-fd8569230ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:31.637688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71df0b0-82ed-498f-ad47-fd8569230ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:31.639815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71df0b0-82ed-498f-ad47-fd8569230ef4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:31.677895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 863/1748] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:31 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:31.679456 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7148f3-2161-46d8-8f90-ec5771405d98 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:31.697629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7148f3-2161-46d8-8f90-ec5771405d98 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:31.708231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9c29c-2273-4b46-a7bc-b247fe28903e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:31.709746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9c29c-2273-4b46-a7bc-b247fe28903e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:31.710315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9c29c-2273-4b46-a7bc-b247fe28903e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:31.710854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9c29c-2273-4b46-a7bc-b247fe28903e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:31.713256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9c29c-2273-4b46-a7bc-b247fe28903e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:31.715869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9c29c-2273-4b46-a7bc-b247fe28903e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:31.743300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 886/1749] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:31.747602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71df0b0-82ed-498f-ad47-fd8569230ef4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:31.758505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71df0b0-82ed-498f-ad47-fd8569230ef4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:31.776020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc38d3-2928-4d43-a3ee-dd044bfabe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:31.777536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc38d3-2928-4d43-a3ee-dd044bfabe2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:31.778405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc38d3-2928-4d43-a3ee-dd044bfabe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:31.778892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc38d3-2928-4d43-a3ee-dd044bfabe2e admin admin] PATH_INFO: `/v3/projects/bf75710219054812a762ec8ba875f619/users/fdb5a7d1d5d74440b7b6ce234615885d/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:31.827482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc38d3-2928-4d43-a3ee-dd044bfabe2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf75710219054812a762ec8ba875f619, user_id=fdb5a7d1d5d74440b7b6ce234615885d, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:31.833602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc38d3-2928-4d43-a3ee-dd044bfabe2e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:31.861900 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 864/1750] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:31 2022] PUT /identity/v3/projects/bf75710219054812a762ec8ba875f619/users/fdb5a7d1d5d74440b7b6ce234615885d/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:31.875837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9c29c-2273-4b46-a7bc-b247fe28903e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:31.891710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9c29c-2273-4b46-a7bc-b247fe28903e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:31.948980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:31.987239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebedc85-94d9-40f9-8953-738942d16a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:31.988304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebedc85-94d9-40f9-8953-738942d16a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:31.988756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebedc85-94d9-40f9-8953-738942d16a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:31.989542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebedc85-94d9-40f9-8953-738942d16a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:31.990837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebedc85-94d9-40f9-8953-738942d16a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:31.992710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebedc85-94d9-40f9-8953-738942d16a0b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:32.007407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 887/1751] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:32.011529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc38d3-2928-4d43-a3ee-dd044bfabe2e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:32.023525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc38d3-2928-4d43-a3ee-dd044bfabe2e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:32.031095 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6c4b1-6ba5-420b-a157-505b06c8e6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:32.033502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6c4b1-6ba5-420b-a157-505b06c8e6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:32.034112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6c4b1-6ba5-420b-a157-505b06c8e6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:32.034972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6c4b1-6ba5-420b-a157-505b06c8e6ee admin admin] PATH_INFO: `/v3/projects/bf75710219054812a762ec8ba875f619/users/fdb5a7d1d5d74440b7b6ce234615885d/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:32.040860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6c4b1-6ba5-420b-a157-505b06c8e6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf75710219054812a762ec8ba875f619, user_id=fdb5a7d1d5d74440b7b6ce234615885d, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:32.046585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6c4b1-6ba5-420b-a157-505b06c8e6ee admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:32.076170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 865/1752] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:32 2022] PUT /identity/v3/projects/bf75710219054812a762ec8ba875f619/users/fdb5a7d1d5d74440b7b6ce234615885d/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:32.078892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a6d4b-984f-47b8-961c-eac9bc92632e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:32.079415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a6d4b-984f-47b8-961c-eac9bc92632e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:32.080451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a6d4b-984f-47b8-961c-eac9bc92632e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:32.154828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9e1a6d4b-984f-47b8-961c-eac9bc92632e None None] MFA Rules not processed for user `fdb5a7d1d5d74440b7b6ce234615885d`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:32.157282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9e1a6d4b-984f-47b8-961c-eac9bc92632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:32.279389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 888/1753] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:32.294002 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6c4b1-6ba5-420b-a157-505b06c8e6ee admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:32.297192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0ea6c4b1-6ba5-420b-a157-505b06c8e6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:32.432265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6c4b1-6ba5-420b-a157-505b06c8e6ee admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:32.438478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d6177-099f-460b-b7bb-c7b2adb5fbb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:32.439686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d6177-099f-460b-b7bb-c7b2adb5fbb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:32.440178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d6177-099f-460b-b7bb-c7b2adb5fbb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:32.440646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d6177-099f-460b-b7bb-c7b2adb5fbb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:32.460375 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d6177-099f-460b-b7bb-c7b2adb5fbb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:32.463018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d6177-099f-460b-b7bb-c7b2adb5fbb3 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:32.482251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 866/1754] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:33.309033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a6d4b-984f-47b8-961c-eac9bc92632e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:33.386643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a6d4b-984f-47b8-961c-eac9bc92632e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:33.393095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:33.394091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:33.394415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:33.394697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] PATH_INFO: `/v3/users/b6ac835d50a14333bd10f9cc6c02c937` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:33.396274 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ac835d50a14333bd10f9cc6c02c937)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:33.398321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:33.473651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ac835d50a14333bd10f9cc6c02c937'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:33.474647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ac835d50a14333bd10f9cc6c02c937'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:33.507001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ac835d50a14333bd10f9cc6c02c937'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:33.526243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:33.545116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ac835d50a14333bd10f9cc6c02c937'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:33.559907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ac835d50a14333bd10f9cc6c02c937'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:33.568620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 889/1755] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:33 2022] DELETE /identity/v3/users/b6ac835d50a14333bd10f9cc6c02c937 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:33.573742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43703d-21f5-431b-927f-cd2120c85445 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:33.574645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43703d-21f5-431b-927f-cd2120c85445 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:33.574645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43703d-21f5-431b-927f-cd2120c85445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:33.594238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1d43703d-21f5-431b-927f-cd2120c85445 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:33.596078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1d43703d-21f5-431b-927f-cd2120c85445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:33.689215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 867/1756] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:33.705283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:33.707752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:33.783863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cca8f-0300-4855-b4c9-20cc03c3bac9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:33.788375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2ca4b-878a-434e-a7c4-f2e6fc450234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:33.789152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2ca4b-878a-434e-a7c4-f2e6fc450234 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:33.789535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2ca4b-878a-434e-a7c4-f2e6fc450234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:33.789818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2ca4b-878a-434e-a7c4-f2e6fc450234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:33.801257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2ca4b-878a-434e-a7c4-f2e6fc450234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:33.802788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2ca4b-878a-434e-a7c4-f2e6fc450234 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:33.813588 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 890/1757] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:34.127227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43703d-21f5-431b-927f-cd2120c85445 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:34.128844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1d43703d-21f5-431b-927f-cd2120c85445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:34.143281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43703d-21f5-431b-927f-cd2120c85445 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:34.148271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:34.149305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:34.149694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:34.150204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] PATH_INFO: `/v3/projects/6372c972cbde4782bc80760a9a671315` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:34.151814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6372c972cbde4782bc80760a9a671315)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:34.153948 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:34.183116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:34.211057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6372c972cbde4782bc80760a9a671315', 'request_id': 'req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.217764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6372c972cbde4782bc80760a9a671315', 'request_id': 'req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.224559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6372c972cbde4782bc80760a9a671315', 'request_id': 'req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.241044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] The token cache is being invalidate because project 6372c972cbde4782bc80760a9a671315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:34.241344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.242182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.243835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 868/1758] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:34 2022] DELETE /identity/v3/projects/6372c972cbde4782bc80760a9a671315 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:34.248020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2ca4b-878a-434e-a7c4-f2e6fc450234 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:34.259723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2ca4b-878a-434e-a7c4-f2e6fc450234 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:34.340902 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:34.341836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:34.342273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:34.342557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] PATH_INFO: `/v3/users/fdb5a7d1d5d74440b7b6ce234615885d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:34.343882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb5a7d1d5d74440b7b6ce234615885d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:34.345887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:34.402890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb5a7d1d5d74440b7b6ce234615885d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.403709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb5a7d1d5d74440b7b6ce234615885d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.416467 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb5a7d1d5d74440b7b6ce234615885d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.430848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb5a7d1d5d74440b7b6ce234615885d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.438793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fc85d036-32af-42ed-90a8-8c82ddcf88a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb5a7d1d5d74440b7b6ce234615885d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.441600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:34.446116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 891/1759] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:34 2022] DELETE /identity/v3/users/fdb5a7d1d5d74440b7b6ce234615885d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:34.792674 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:34.795208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:34.812061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf0e58-3eb6-4283-b7c5-2c6ae7954ac5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:34.891190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:34.892149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:34.892445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:34.892733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] PATH_INFO: `/v3/projects/bf75710219054812a762ec8ba875f619` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:34.893973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf75710219054812a762ec8ba875f619)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:34.895997 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:34.946793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf75710219054812a762ec8ba875f619', 'request_id': 'req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.953372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf75710219054812a762ec8ba875f619', 'request_id': 'req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.959558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf75710219054812a762ec8ba875f619', 'request_id': 'req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.973686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] The token cache is being invalidate because project bf75710219054812a762ec8ba875f619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:34.974012 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.974716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f3d5ae4-3d1c-486c-9cc5-d2a1409d5ca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:34.976435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 869/1760] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:34 2022] DELETE /identity/v3/projects/bf75710219054812a762ec8ba875f619 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:34.981679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dde92-f076-40b5-971d-a7c65d5ea309 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:34.981981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dde92-f076-40b5-971d-a7c65d5ea309 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:34.982276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dde92-f076-40b5-971d-a7c65d5ea309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.003495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-171dde92-f076-40b5-971d-a7c65d5ea309 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:35.004781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-171dde92-f076-40b5-971d-a7c65d5ea309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:35.073015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 892/1761] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:35.081321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b260de-7d42-4d1f-ab99-d7d3ef7144c1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.081622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b260de-7d42-4d1f-ab99-d7d3ef7144c1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.081912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b260de-7d42-4d1f-ab99-d7d3ef7144c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.102960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-40b260de-7d42-4d1f-ab99-d7d3ef7144c1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:35.103699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-40b260de-7d42-4d1f-ab99-d7d3ef7144c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:35.113463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 870/1762] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:35.118433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dde92-f076-40b5-971d-a7c65d5ea309 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:35.128943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dde92-f076-40b5-971d-a7c65d5ea309 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:35.133633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb56686-4854-4ade-ad06-5819fff22517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:35.134317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb56686-4854-4ade-ad06-5819fff22517 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.134603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb56686-4854-4ade-ad06-5819fff22517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.134882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb56686-4854-4ade-ad06-5819fff22517 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.135680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb56686-4854-4ade-ad06-5819fff22517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:35.135987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb56686-4854-4ade-ad06-5819fff22517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:35.137616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb56686-4854-4ade-ad06-5819fff22517 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:35.150499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 893/1763] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:35.154841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b260de-7d42-4d1f-ab99-d7d3ef7144c1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:35.165372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b260de-7d42-4d1f-ab99-d7d3ef7144c1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:35.170174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4099a93-207c-4a09-b639-ef148ad127bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:35.170866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099a93-207c-4a09-b639-ef148ad127bf admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.171313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099a93-207c-4a09-b639-ef148ad127bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.171623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099a93-207c-4a09-b639-ef148ad127bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.172341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4099a93-207c-4a09-b639-ef148ad127bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:35.174931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4099a93-207c-4a09-b639-ef148ad127bf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:35.202881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 871/1764] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:35 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:35.207703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb56686-4854-4ade-ad06-5819fff22517 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:35.218223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb56686-4854-4ade-ad06-5819fff22517 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:35.282079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595845e-466d-4e0f-bce6-9ce32c57d69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:35.283363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595845e-466d-4e0f-bce6-9ce32c57d69c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.283653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595845e-466d-4e0f-bce6-9ce32c57d69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.283931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595845e-466d-4e0f-bce6-9ce32c57d69c admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.284654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595845e-466d-4e0f-bce6-9ce32c57d69c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:35.286734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595845e-466d-4e0f-bce6-9ce32c57d69c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:35.308547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 894/1765] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:35 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:35.312980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4099a93-207c-4a09-b639-ef148ad127bf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:35.324238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4099a93-207c-4a09-b639-ef148ad127bf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:35.329093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6a869-ff94-404b-80e2-65d52d8deebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:35.330541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6a869-ff94-404b-80e2-65d52d8deebe admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.330831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6a869-ff94-404b-80e2-65d52d8deebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.331110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6a869-ff94-404b-80e2-65d52d8deebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.331930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6a869-ff94-404b-80e2-65d52d8deebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:35.333596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6a869-ff94-404b-80e2-65d52d8deebe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:35.348311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 872/1766] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:35.348311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:35.351241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595845e-466d-4e0f-bce6-9ce32c57d69c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:35.365127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595845e-466d-4e0f-bce6-9ce32c57d69c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:35.370250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b870-6d82-4033-8547-fea12ff0d648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:35.370964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b870-6d82-4033-8547-fea12ff0d648 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.371252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b870-6d82-4033-8547-fea12ff0d648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.371768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b870-6d82-4033-8547-fea12ff0d648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.373400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22b870-6d82-4033-8547-fea12ff0d648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:35.375855 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22b870-6d82-4033-8547-fea12ff0d648 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:35.390844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 895/1767] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:35.392800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6a869-ff94-404b-80e2-65d52d8deebe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:35.406130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6a869-ff94-404b-80e2-65d52d8deebe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:35.411891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddcff3-b320-4d79-b4f9-350249f9ab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:35.413984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddcff3-b320-4d79-b4f9-350249f9ab60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.413984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddcff3-b320-4d79-b4f9-350249f9ab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.414127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddcff3-b320-4d79-b4f9-350249f9ab60 admin admin] PATH_INFO: `/v3/projects/ca5266dcc81440c798dffce12d9dfcd9/users/23b76ce486c9410fbad7564e11d98dad/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.435448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddcff3-b320-4d79-b4f9-350249f9ab60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5266dcc81440c798dffce12d9dfcd9, user_id=23b76ce486c9410fbad7564e11d98dad, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:35.438281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddcff3-b320-4d79-b4f9-350249f9ab60 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:35.456288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 873/1768] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:35 2022] PUT /identity/v3/projects/ca5266dcc81440c798dffce12d9dfcd9/users/23b76ce486c9410fbad7564e11d98dad/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:35.472554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814be4-780e-4cd0-99dd-73ee481ca33b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.472997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814be4-780e-4cd0-99dd-73ee481ca33b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.473421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814be4-780e-4cd0-99dd-73ee481ca33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.514386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b9814be4-780e-4cd0-99dd-73ee481ca33b None None] MFA Rules not processed for user `23b76ce486c9410fbad7564e11d98dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:35.515726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b9814be4-780e-4cd0-99dd-73ee481ca33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:35.587556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:35.604385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 896/1769] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:35.612316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddcff3-b320-4d79-b4f9-350249f9ab60 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:35.614288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21ddcff3-b320-4d79-b4f9-350249f9ab60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:35.672854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddcff3-b320-4d79-b4f9-350249f9ab60 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:35.676576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200b400-7281-4bf5-9050-6b22531f930d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:35.677244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200b400-7281-4bf5-9050-6b22531f930d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.677532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200b400-7281-4bf5-9050-6b22531f930d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.677813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200b400-7281-4bf5-9050-6b22531f930d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.687586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200b400-7281-4bf5-9050-6b22531f930d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:35.689587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200b400-7281-4bf5-9050-6b22531f930d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:35.699721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 874/1770] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:35.719978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9814be4-780e-4cd0-99dd-73ee481ca33b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:35.792261 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9814be4-780e-4cd0-99dd-73ee481ca33b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:35.797257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:35.798210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.798474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.798757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] PATH_INFO: `/v3/users/23b76ce486c9410fbad7564e11d98dad` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.800127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] RBAC: Authorizing `identity:delete_user(user_id=23b76ce486c9410fbad7564e11d98dad)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:35.802152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:35.856256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23b76ce486c9410fbad7564e11d98dad'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:35.857090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23b76ce486c9410fbad7564e11d98dad'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:35.866690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23b76ce486c9410fbad7564e11d98dad'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:35.884320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23b76ce486c9410fbad7564e11d98dad'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:35.896382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23b76ce486c9410fbad7564e11d98dad'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:35.903719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 897/1771] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:35 2022] DELETE /identity/v3/users/23b76ce486c9410fbad7564e11d98dad => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:35.908555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad403-3269-4912-b257-480334cb54fd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:35.910598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad403-3269-4912-b257-480334cb54fd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:35.911044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad403-3269-4912-b257-480334cb54fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:35.931511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c76ad403-3269-4912-b257-480334cb54fd None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:35.932787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c76ad403-3269-4912-b257-480334cb54fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:36.025529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 875/1772] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:36.052475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:36.056391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:36.168600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48783f38-3b73-4b78-93a4-5b4f9ef0d86e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:36.173780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166a4d4-9471-4b4b-a6db-f1b16af74979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:36.174463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166a4d4-9471-4b4b-a6db-f1b16af74979 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:36.175292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166a4d4-9471-4b4b-a6db-f1b16af74979 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:36.175292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166a4d4-9471-4b4b-a6db-f1b16af74979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:36.185357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166a4d4-9471-4b4b-a6db-f1b16af74979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:36.187006 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166a4d4-9471-4b4b-a6db-f1b16af74979 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:36.205188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 898/1773] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:37.996055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ad403-3269-4912-b257-480334cb54fd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:37.997717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c76ad403-3269-4912-b257-480334cb54fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:38.022614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ad403-3269-4912-b257-480334cb54fd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:38.040906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:38.042397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:38.044048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:38.044358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] PATH_INFO: `/v3/projects/ca5266dcc81440c798dffce12d9dfcd9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:38.052127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5266dcc81440c798dffce12d9dfcd9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:38.059159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:38.200917 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5266dcc81440c798dffce12d9dfcd9', 'request_id': 'req-b8f01018-e8b8-4026-9633-49318da0c609'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:38.224497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5266dcc81440c798dffce12d9dfcd9', 'request_id': 'req-b8f01018-e8b8-4026-9633-49318da0c609'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:38.235282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5266dcc81440c798dffce12d9dfcd9', 'request_id': 'req-b8f01018-e8b8-4026-9633-49318da0c609'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:38.270741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] The token cache is being invalidate because project ca5266dcc81440c798dffce12d9dfcd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:38.271041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:38.273383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b8f01018-e8b8-4026-9633-49318da0c609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:38.283649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 876/1774] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:37 2022] DELETE /identity/v3/projects/ca5266dcc81440c798dffce12d9dfcd9 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:38.286914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef442a3-2844-49be-b498-d88ed1739982 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:38.287853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef442a3-2844-49be-b498-d88ed1739982 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:38.289498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef442a3-2844-49be-b498-d88ed1739982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:38.311511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1ef442a3-2844-49be-b498-d88ed1739982 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:38.312998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1ef442a3-2844-49be-b498-d88ed1739982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:38.332536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:38.492995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 899/1775] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:38.493794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3729c7-53c6-4036-9b54-88cc104ef14e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:38.494096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3729c7-53c6-4036-9b54-88cc104ef14e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:38.494388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3729c7-53c6-4036-9b54-88cc104ef14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:38.507789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:38.546154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7a3729c7-53c6-4036-9b54-88cc104ef14e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:38.547459 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7a3729c7-53c6-4036-9b54-88cc104ef14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:38.570176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 877/1776] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:38.574307 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef442a3-2844-49be-b498-d88ed1739982 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:38.587949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef442a3-2844-49be-b498-d88ed1739982 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:38.633025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bfd04-d303-467b-a56f-64f385b94da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:38.633736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bfd04-d303-467b-a56f-64f385b94da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:38.634285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bfd04-d303-467b-a56f-64f385b94da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:38.634385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bfd04-d303-467b-a56f-64f385b94da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:38.635198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bfd04-d303-467b-a56f-64f385b94da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:38.638566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bfd04-d303-467b-a56f-64f385b94da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:38.640435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bfd04-d303-467b-a56f-64f385b94da8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:38.669427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3729c7-53c6-4036-9b54-88cc104ef14e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:38.671976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 900/1777] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:38.689537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3729c7-53c6-4036-9b54-88cc104ef14e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:38.706049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a821137-516e-442c-8481-6463c27baede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:38.708023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a821137-516e-442c-8481-6463c27baede admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:38.708323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a821137-516e-442c-8481-6463c27baede admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:38.709724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a821137-516e-442c-8481-6463c27baede admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:38.713240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a821137-516e-442c-8481-6463c27baede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:38.716876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a821137-516e-442c-8481-6463c27baede admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:38.819871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 878/1778] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:38 2022] POST /identity/v3/projects => generated 359 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:38.822880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bfd04-d303-467b-a56f-64f385b94da8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:38.844044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bfd04-d303-467b-a56f-64f385b94da8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:38.982989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a821137-516e-442c-8481-6463c27baede admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:38.986284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7a821137-516e-442c-8481-6463c27baede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:38.993408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5ea21-cceb-4a5f-8f00-8d4488e0a25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:38.994368 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5ea21-cceb-4a5f-8f00-8d4488e0a25a admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:38.994666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5ea21-cceb-4a5f-8f00-8d4488e0a25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:38.994955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5ea21-cceb-4a5f-8f00-8d4488e0a25a admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:38.996114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5ea21-cceb-4a5f-8f00-8d4488e0a25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:38.998890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5ea21-cceb-4a5f-8f00-8d4488e0a25a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:39.005216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a821137-516e-442c-8481-6463c27baede admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:39.011800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:39.013424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.014161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.014553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] PATH_INFO: `/v3/users/a06f477c35004033a3d13511e5eb612c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.016598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06f477c35004033a3d13511e5eb612c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:39.026017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:39.043218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 901/1779] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:38 2022] POST /identity/v3/users => generated 280 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:39.049737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5ea21-cceb-4a5f-8f00-8d4488e0a25a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:39.069863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5ea21-cceb-4a5f-8f00-8d4488e0a25a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:39.075138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e756a-2c1d-40eb-a2e8-6fb7154a68c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:39.076262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e756a-2c1d-40eb-a2e8-6fb7154a68c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.077298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e756a-2c1d-40eb-a2e8-6fb7154a68c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.077581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e756a-2c1d-40eb-a2e8-6fb7154a68c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.078358 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e756a-2c1d-40eb-a2e8-6fb7154a68c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:39.080438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e756a-2c1d-40eb-a2e8-6fb7154a68c3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:39.098436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 902/1780] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:39.100292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e756a-2c1d-40eb-a2e8-6fb7154a68c3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:39.113132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e756a-2c1d-40eb-a2e8-6fb7154a68c3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:39.118360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea0306-7456-455b-b61b-434cf160d5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:39.119043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea0306-7456-455b-b61b-434cf160d5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.120118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea0306-7456-455b-b61b-434cf160d5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.120427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea0306-7456-455b-b61b-434cf160d5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.121175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea0306-7456-455b-b61b-434cf160d5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:39.122213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06f477c35004033a3d13511e5eb612c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.122812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea0306-7456-455b-b61b-434cf160d5cb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:39.133471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06f477c35004033a3d13511e5eb612c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.139158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 903/1781] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:39.140842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea0306-7456-455b-b61b-434cf160d5cb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:39.144656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06f477c35004033a3d13511e5eb612c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.152909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea0306-7456-455b-b61b-434cf160d5cb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:39.166210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08797f5-b4aa-4d21-86c9-ff80de153de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:39.167160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08797f5-b4aa-4d21-86c9-ff80de153de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.167513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08797f5-b4aa-4d21-86c9-ff80de153de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.167837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08797f5-b4aa-4d21-86c9-ff80de153de5 admin admin] PATH_INFO: `/v3/projects/903c6398d40a47fcb29aed17dd448db0/users/700c655027d4435fb08a5709d2fe7d1c/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.169415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06f477c35004033a3d13511e5eb612c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.170334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ec684074-3fbd-4a49-b70a-827292cfb46f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06f477c35004033a3d13511e5eb612c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.178762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 879/1782] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:38 2022] DELETE /identity/v3/users/a06f477c35004033a3d13511e5eb612c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:39.183688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f594c24-8609-4e4c-a113-0cf1d6a79ca7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.184276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f594c24-8609-4e4c-a113-0cf1d6a79ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.185298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f594c24-8609-4e4c-a113-0cf1d6a79ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.194994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08797f5-b4aa-4d21-86c9-ff80de153de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903c6398d40a47fcb29aed17dd448db0, user_id=700c655027d4435fb08a5709d2fe7d1c, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:39.197806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08797f5-b4aa-4d21-86c9-ff80de153de5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:39.224236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5f594c24-8609-4e4c-a113-0cf1d6a79ca7 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:39.226074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5f594c24-8609-4e4c-a113-0cf1d6a79ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:39.227467 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 904/1783] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:39 2022] PUT /identity/v3/projects/903c6398d40a47fcb29aed17dd448db0/users/700c655027d4435fb08a5709d2fe7d1c/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:39.234628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466f18c-a9a9-4865-bfc0-ab2065ca3ca5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.235167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466f18c-a9a9-4865-bfc0-ab2065ca3ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.235546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466f18c-a9a9-4865-bfc0-ab2065ca3ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.269053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1466f18c-a9a9-4865-bfc0-ab2065ca3ca5 None None] MFA Rules not processed for user `700c655027d4435fb08a5709d2fe7d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:39.270213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1466f18c-a9a9-4865-bfc0-ab2065ca3ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:39.319607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:39.320739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 880/1784] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:39.335340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f594c24-8609-4e4c-a113-0cf1d6a79ca7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:39.338144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5f594c24-8609-4e4c-a113-0cf1d6a79ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:39.371233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 905/1785] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:39.421938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466f18c-a9a9-4865-bfc0-ab2065ca3ca5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:39.424396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1466f18c-a9a9-4865-bfc0-ab2065ca3ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:39.429518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:39.446539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f594c24-8609-4e4c-a113-0cf1d6a79ca7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:39.456784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56b94d-8efb-4ac7-b5c8-1c36d8cb26a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:39.457471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56b94d-8efb-4ac7-b5c8-1c36d8cb26a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.457764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56b94d-8efb-4ac7-b5c8-1c36d8cb26a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.458048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56b94d-8efb-4ac7-b5c8-1c36d8cb26a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.473619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56b94d-8efb-4ac7-b5c8-1c36d8cb26a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:39.475200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56b94d-8efb-4ac7-b5c8-1c36d8cb26a6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:39.489486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 881/1786] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:39.501789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466f18c-a9a9-4865-bfc0-ab2065ca3ca5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:39.508717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbea55f-2214-42a7-94ba-3212b701946e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:39.510271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea55f-2214-42a7-94ba-3212b701946e service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.510271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea55f-2214-42a7-94ba-3212b701946e service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.510663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea55f-2214-42a7-94ba-3212b701946e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.523869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbea55f-2214-42a7-94ba-3212b701946e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:39.525506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbea55f-2214-42a7-94ba-3212b701946e service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:39.538499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 906/1787] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:39 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:39.555377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56b94d-8efb-4ac7-b5c8-1c36d8cb26a6 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:39.556142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2a56b94d-8efb-4ac7-b5c8-1c36d8cb26a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:39.571386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56b94d-8efb-4ac7-b5c8-1c36d8cb26a6 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:39.578529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:39.580339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.580578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.580930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] PATH_INFO: `/v3/users/700c655027d4435fb08a5709d2fe7d1c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.582417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=700c655027d4435fb08a5709d2fe7d1c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:39.584527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:39.657074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '700c655027d4435fb08a5709d2fe7d1c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.672789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '700c655027d4435fb08a5709d2fe7d1c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.694860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '700c655027d4435fb08a5709d2fe7d1c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.721442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '700c655027d4435fb08a5709d2fe7d1c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.722638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '700c655027d4435fb08a5709d2fe7d1c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:39.747791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 882/1788] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:39 2022] DELETE /identity/v3/users/700c655027d4435fb08a5709d2fe7d1c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:39.752724 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2da3-0394-4633-a6be-d9be0d2bc543 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:39.752909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2da3-0394-4633-a6be-d9be0d2bc543 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:39.753182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2da3-0394-4633-a6be-d9be0d2bc543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:39.774396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e76c2da3-0394-4633-a6be-d9be0d2bc543 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:39.776378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e76c2da3-0394-4633-a6be-d9be0d2bc543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:39.879716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 907/1789] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:39.919471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:39.921088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:40.018678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40073c66-e180-4521-ad16-b10680982b15 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:40.037180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5be44c-a4fc-4d01-b74e-138ae97680d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:40.039464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5be44c-a4fc-4d01-b74e-138ae97680d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:40.040385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5be44c-a4fc-4d01-b74e-138ae97680d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:40.041294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5be44c-a4fc-4d01-b74e-138ae97680d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:40.075692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5be44c-a4fc-4d01-b74e-138ae97680d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:40.077709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5be44c-a4fc-4d01-b74e-138ae97680d5 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:40.094999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 883/1790] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:40.621525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c2da3-0394-4633-a6be-d9be0d2bc543 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:40.623251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e76c2da3-0394-4633-a6be-d9be0d2bc543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:40.649095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c2da3-0394-4633-a6be-d9be0d2bc543 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:40.661255 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:40.663835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:40.664433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:40.664900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] PATH_INFO: `/v3/projects/a3f3d6309ebc43ba8b8d294b2615e564` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:40.667198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f3d6309ebc43ba8b8d294b2615e564)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:40.670605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:40.767283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f3d6309ebc43ba8b8d294b2615e564', 'request_id': 'req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:40.774916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f3d6309ebc43ba8b8d294b2615e564', 'request_id': 'req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:40.787096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f3d6309ebc43ba8b8d294b2615e564', 'request_id': 'req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:40.808749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] The token cache is being invalidate because project a3f3d6309ebc43ba8b8d294b2615e564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:40.809092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:40.809775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:40.811835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 908/1791] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:40 2022] DELETE /identity/v3/projects/a3f3d6309ebc43ba8b8d294b2615e564 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:40.816116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5be44c-a4fc-4d01-b74e-138ae97680d5 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:40.831206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5be44c-a4fc-4d01-b74e-138ae97680d5 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:40.947920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:40.949253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:40.949950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:40.950440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] PATH_INFO: `/v3/users/65a7d27f33a54080880ebc724dabdf59` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:40.952314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a7d27f33a54080880ebc724dabdf59)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:40.954956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:40.962354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:41.097123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a7d27f33a54080880ebc724dabdf59'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.135722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a7d27f33a54080880ebc724dabdf59'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.157001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:41.159018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:41.167770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a7d27f33a54080880ebc724dabdf59'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.177558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:41.186841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaba58-c770-45fb-8d4d-0a5c0ccf691b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:41.193352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a7d27f33a54080880ebc724dabdf59'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.194599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c8b406c-f28a-48bb-b1d9-c2b16fc041ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a7d27f33a54080880ebc724dabdf59'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.196654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:41.197678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:41.197975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:41.198375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] PATH_INFO: `/v3/users/372b0956ae414ec1bc8b1fdd2fcdcf47` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:41.199663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=372b0956ae414ec1bc8b1fdd2fcdcf47)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:41.203071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:41.209930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 884/1792] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:40 2022] DELETE /identity/v3/users/65a7d27f33a54080880ebc724dabdf59 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:41.318695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372b0956ae414ec1bc8b1fdd2fcdcf47'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.319914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372b0956ae414ec1bc8b1fdd2fcdcf47'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.334814 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372b0956ae414ec1bc8b1fdd2fcdcf47'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.361311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372b0956ae414ec1bc8b1fdd2fcdcf47'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.387481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372b0956ae414ec1bc8b1fdd2fcdcf47'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:41.412250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 909/1793] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:41 2022] DELETE /identity/v3/users/372b0956ae414ec1bc8b1fdd2fcdcf47 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:41.418547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef96b31-5013-430e-8d1a-2d999761f77e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:41.418897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef96b31-5013-430e-8d1a-2d999761f77e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:41.419190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef96b31-5013-430e-8d1a-2d999761f77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:41.458426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3ef96b31-5013-430e-8d1a-2d999761f77e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:41.460063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ef96b31-5013-430e-8d1a-2d999761f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:41.673632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 885/1794] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:41.695920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:41.698812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:41.835119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6944e4c-d597-48a9-811c-12d06a0c05d7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:41.843233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52cdf7-2c23-4ca9-8795-4971f0ff403e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:41.844628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52cdf7-2c23-4ca9-8795-4971f0ff403e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:41.844628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52cdf7-2c23-4ca9-8795-4971f0ff403e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:41.844628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52cdf7-2c23-4ca9-8795-4971f0ff403e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:41.858496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52cdf7-2c23-4ca9-8795-4971f0ff403e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:41.862301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52cdf7-2c23-4ca9-8795-4971f0ff403e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:41.896829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 910/1795] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:41.913278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef96b31-5013-430e-8d1a-2d999761f77e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:41.920256 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ef96b31-5013-430e-8d1a-2d999761f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:41.957350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef96b31-5013-430e-8d1a-2d999761f77e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:41.966015 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:41.967706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:41.968300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:41.968849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] PATH_INFO: `/v3/projects/903c6398d40a47fcb29aed17dd448db0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:41.970685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=903c6398d40a47fcb29aed17dd448db0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:41.973514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:42.099702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903c6398d40a47fcb29aed17dd448db0', 'request_id': 'req-e0ef9bb0-8589-42db-9306-3b86bf90d228'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:42.110271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903c6398d40a47fcb29aed17dd448db0', 'request_id': 'req-e0ef9bb0-8589-42db-9306-3b86bf90d228'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:42.122521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903c6398d40a47fcb29aed17dd448db0', 'request_id': 'req-e0ef9bb0-8589-42db-9306-3b86bf90d228'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:42.148715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] The token cache is being invalidate because project 903c6398d40a47fcb29aed17dd448db0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:42.149153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:42.150025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e0ef9bb0-8589-42db-9306-3b86bf90d228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:42.152891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 886/1796] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:41 2022] DELETE /identity/v3/projects/903c6398d40a47fcb29aed17dd448db0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:42.159861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e3445-0c49-4d9b-9251-315f051bedf2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:42.160580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e3445-0c49-4d9b-9251-315f051bedf2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:42.161236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e3445-0c49-4d9b-9251-315f051bedf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:42.208513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ec3e3445-0c49-4d9b-9251-315f051bedf2 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:42.212127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ec3e3445-0c49-4d9b-9251-315f051bedf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:42.408866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 911/1797] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:42.420605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e45a-dca2-4879-b40d-b786958bfae0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:42.420989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e45a-dca2-4879-b40d-b786958bfae0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:42.421230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e45a-dca2-4879-b40d-b786958bfae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:42.452392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:42.480362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d602e45a-dca2-4879-b40d-b786958bfae0 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:42.481718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d602e45a-dca2-4879-b40d-b786958bfae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:42.509745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e3445-0c49-4d9b-9251-315f051bedf2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:42.510515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 887/1798] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:42.524982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e3445-0c49-4d9b-9251-315f051bedf2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:42.533367 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337f203-96b7-4eea-91f9-93f4df37b265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:42.534569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337f203-96b7-4eea-91f9-93f4df37b265 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:42.535371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337f203-96b7-4eea-91f9-93f4df37b265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:42.536160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337f203-96b7-4eea-91f9-93f4df37b265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:42.537284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337f203-96b7-4eea-91f9-93f4df37b265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:42.537802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337f203-96b7-4eea-91f9-93f4df37b265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:42.539910 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337f203-96b7-4eea-91f9-93f4df37b265 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:42.572968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 912/1799] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:42.579017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602e45a-dca2-4879-b40d-b786958bfae0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:42.629130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602e45a-dca2-4879-b40d-b786958bfae0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:42.646731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa3ff0-0bf8-450d-84b8-d0a3314cebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:42.648181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa3ff0-0bf8-450d-84b8-d0a3314cebfc admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:42.648710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa3ff0-0bf8-450d-84b8-d0a3314cebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:42.655687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa3ff0-0bf8-450d-84b8-d0a3314cebfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:42.656987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa3ff0-0bf8-450d-84b8-d0a3314cebfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:42.661218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa3ff0-0bf8-450d-84b8-d0a3314cebfc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:42.730361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 888/1800] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:42 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:42.730775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337f203-96b7-4eea-91f9-93f4df37b265 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:42.743486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337f203-96b7-4eea-91f9-93f4df37b265 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:42.748739 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:42.854484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debc49d-53ed-4921-b813-877fb71e6258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:42.855928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc49d-53ed-4921-b813-877fb71e6258 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:42.856574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc49d-53ed-4921-b813-877fb71e6258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:42.857251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc49d-53ed-4921-b813-877fb71e6258 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:42.858359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2debc49d-53ed-4921-b813-877fb71e6258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:42.860580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2debc49d-53ed-4921-b813-877fb71e6258 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:42.898211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 913/1801] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:42 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:42.899788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa3ff0-0bf8-450d-84b8-d0a3314cebfc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:42.921316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa3ff0-0bf8-450d-84b8-d0a3314cebfc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:42.929721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215291e-1fe8-466e-bc8e-4d0e76ea95d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:42.930760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215291e-1fe8-466e-bc8e-4d0e76ea95d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:42.931142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215291e-1fe8-466e-bc8e-4d0e76ea95d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:42.931472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215291e-1fe8-466e-bc8e-4d0e76ea95d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:42.932260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215291e-1fe8-466e-bc8e-4d0e76ea95d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:42.933975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215291e-1fe8-466e-bc8e-4d0e76ea95d1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:42.969063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 889/1802] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:42.974633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debc49d-53ed-4921-b813-877fb71e6258 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:42.992426 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debc49d-53ed-4921-b813-877fb71e6258 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:42.998825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfb03c-a171-4bc1-902b-951be265b0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:42.999624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfb03c-a171-4bc1-902b-951be265b0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:42.999916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfb03c-a171-4bc1-902b-951be265b0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:43.000196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfb03c-a171-4bc1-902b-951be265b0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:43.000946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfb03c-a171-4bc1-902b-951be265b0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:43.002595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfb03c-a171-4bc1-902b-951be265b0c3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:43.031327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 914/1803] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:43.036840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215291e-1fe8-466e-bc8e-4d0e76ea95d1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:43.050987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215291e-1fe8-466e-bc8e-4d0e76ea95d1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:43.062114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd762b35-225e-4db1-b7b4-42630e4a00a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:43.063283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd762b35-225e-4db1-b7b4-42630e4a00a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:43.064317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd762b35-225e-4db1-b7b4-42630e4a00a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:43.064720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd762b35-225e-4db1-b7b4-42630e4a00a7 admin admin] PATH_INFO: `/v3/projects/dba5fa0382bc4a69ab7732ea8171c09b/users/a1710cca0d84444d9b90a90eaf7c55ef/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:43.084276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd762b35-225e-4db1-b7b4-42630e4a00a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba5fa0382bc4a69ab7732ea8171c09b, user_id=a1710cca0d84444d9b90a90eaf7c55ef, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:43.087930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd762b35-225e-4db1-b7b4-42630e4a00a7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:43.092448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfb03c-a171-4bc1-902b-951be265b0c3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:43.094738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a3bfb03c-a171-4bc1-902b-951be265b0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:43.123176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfb03c-a171-4bc1-902b-951be265b0c3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:43.127586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 890/1804] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:43 2022] PUT /identity/v3/projects/dba5fa0382bc4a69ab7732ea8171c09b/users/a1710cca0d84444d9b90a90eaf7c55ef/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:43.140560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87884311-2b9e-40a5-b4aa-62b5eddabe44 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:43.141009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87884311-2b9e-40a5-b4aa-62b5eddabe44 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:43.141615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87884311-2b9e-40a5-b4aa-62b5eddabe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:43.196304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-87884311-2b9e-40a5-b4aa-62b5eddabe44 None None] MFA Rules not processed for user `a1710cca0d84444d9b90a90eaf7c55ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:43.198186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-87884311-2b9e-40a5-b4aa-62b5eddabe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:43.257653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:43.258570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:43.258862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:43.259808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] PATH_INFO: `/v3/projects/e74e1943470a4baaa1b0a00d95432073` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:43.265028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e74e1943470a4baaa1b0a00d95432073)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:43.273654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:43.416317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e74e1943470a4baaa1b0a00d95432073', 'request_id': 'req-b88513b9-e526-477f-bbaf-2aff1866fee3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.420743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 891/1805] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:43.421945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87884311-2b9e-40a5-b4aa-62b5eddabe44 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:43.425796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e74e1943470a4baaa1b0a00d95432073', 'request_id': 'req-b88513b9-e526-477f-bbaf-2aff1866fee3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.427139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-87884311-2b9e-40a5-b4aa-62b5eddabe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:43.443662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e74e1943470a4baaa1b0a00d95432073', 'request_id': 'req-b88513b9-e526-477f-bbaf-2aff1866fee3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.459877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87884311-2b9e-40a5-b4aa-62b5eddabe44 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:43.474502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] The token cache is being invalidate because project e74e1943470a4baaa1b0a00d95432073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:43.475352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.476220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.481984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 915/1806] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:43 2022] DELETE /identity/v3/projects/e74e1943470a4baaa1b0a00d95432073 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:43.483417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:43.486109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:43.512500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:43.553161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:43.554639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:43.556758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:43.559081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] PATH_INFO: `/v3/projects/761d03dc41b64911b761681bab034b36` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:43.562809 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=761d03dc41b64911b761681bab034b36)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:43.569583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:43.586742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88513b9-e526-477f-bbaf-2aff1866fee3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:43.600600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9a775-1989-4537-a109-ff10379ce69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:43.601326 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9a775-1989-4537-a109-ff10379ce69f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:43.601618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9a775-1989-4537-a109-ff10379ce69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:43.601899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9a775-1989-4537-a109-ff10379ce69f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:43.605254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9a775-1989-4537-a109-ff10379ce69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:43.608830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9a775-1989-4537-a109-ff10379ce69f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:43.687457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 916/1807] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:43 2022] POST /identity/v3/projects => generated 375 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:43.691501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b5a96-f9f7-401e-a5cb-5ab2d060e866 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:43.692094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b5a96-f9f7-401e-a5cb-5ab2d060e866 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:43.692627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b5a96-f9f7-401e-a5cb-5ab2d060e866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:43.698331 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:43.727476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '761d03dc41b64911b761681bab034b36', 'request_id': 'req-60a4a35c-4057-4494-83c0-0dd7fa1774b9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.743672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '761d03dc41b64911b761681bab034b36', 'request_id': 'req-60a4a35c-4057-4494-83c0-0dd7fa1774b9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.758459 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '761d03dc41b64911b761681bab034b36', 'request_id': 'req-60a4a35c-4057-4494-83c0-0dd7fa1774b9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.765307 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-933b5a96-f9f7-401e-a5cb-5ab2d060e866 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:43.768216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-933b5a96-f9f7-401e-a5cb-5ab2d060e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:43.796488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] The token cache is being invalidate because project 761d03dc41b64911b761681bab034b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:43.797151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.799570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-60a4a35c-4057-4494-83c0-0dd7fa1774b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:43.801860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 892/1808] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:43 2022] DELETE /identity/v3/projects/761d03dc41b64911b761681bab034b36 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:43.803596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e223dde-2687-4932-8301-d918277fdfdf None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:43.804546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e223dde-2687-4932-8301-d918277fdfdf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:43.806393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e223dde-2687-4932-8301-d918277fdfdf None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:43.808716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 893/1809] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:14:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:14:43.809978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e223dde-2687-4932-8301-d918277fdfdf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:43.812195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4e223dde-2687-4932-8301-d918277fdfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:43.955524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e223dde-2687-4932-8301-d918277fdfdf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:43.968048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8021053-cf90-42b3-be05-ecf45aaa02da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:43.968765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021053-cf90-42b3-be05-ecf45aaa02da admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:43.969054 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021053-cf90-42b3-be05-ecf45aaa02da admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:43.969338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021053-cf90-42b3-be05-ecf45aaa02da admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:43.970638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8021053-cf90-42b3-be05-ecf45aaa02da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:43.972798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8021053-cf90-42b3-be05-ecf45aaa02da admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:43.982830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 917/1810] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:43.986176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b5a96-f9f7-401e-a5cb-5ab2d060e866 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:43.990180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-933b5a96-f9f7-401e-a5cb-5ab2d060e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:44.026287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b5a96-f9f7-401e-a5cb-5ab2d060e866 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.029793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 894/1811] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:43 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:44.032621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174dce-d570-487e-a79f-48438b575e55 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.032621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174dce-d570-487e-a79f-48438b575e55 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.032899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174dce-d570-487e-a79f-48438b575e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.049240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.050299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.050596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.050882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] PATH_INFO: `/v3/users/8f133326a87744ce8d83b3fd44e8cc34` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.052393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f133326a87744ce8d83b3fd44e8cc34)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.054822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.067482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-22174dce-d570-487e-a79f-48438b575e55 None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:44.069104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22174dce-d570-487e-a79f-48438b575e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:44.183184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 895/1812] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:14:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:44.184561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aa4fe-29d1-4db7-b3fb-f659390de316 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.184989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aa4fe-29d1-4db7-b3fb-f659390de316 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.185238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aa4fe-29d1-4db7-b3fb-f659390de316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.205193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f133326a87744ce8d83b3fd44e8cc34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:44.206393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f133326a87744ce8d83b3fd44e8cc34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:44.206553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-931aa4fe-29d1-4db7-b3fb-f659390de316 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:44.207784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-931aa4fe-29d1-4db7-b3fb-f659390de316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:44.218139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 896/1813] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:44.220990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aa4fe-29d1-4db7-b3fb-f659390de316 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.223636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-931aa4fe-29d1-4db7-b3fb-f659390de316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:44.225573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f133326a87744ce8d83b3fd44e8cc34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:44.253829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aa4fe-29d1-4db7-b3fb-f659390de316 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.259409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b14ae-cb93-44a7-9ce5-523ba2c063b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.260111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b14ae-cb93-44a7-9ce5-523ba2c063b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.260589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b14ae-cb93-44a7-9ce5-523ba2c063b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.260913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b14ae-cb93-44a7-9ce5-523ba2c063b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.261722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b14ae-cb93-44a7-9ce5-523ba2c063b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.263567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b14ae-cb93-44a7-9ce5-523ba2c063b6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.267530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f133326a87744ce8d83b3fd44e8cc34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:44.279114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f133326a87744ce8d83b3fd44e8cc34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:44.281682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 897/1814] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:44.284225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b14ae-cb93-44a7-9ce5-523ba2c063b6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.287400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 918/1815] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:43 2022] DELETE /identity/v3/users/8f133326a87744ce8d83b3fd44e8cc34 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:44.288644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.290279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:44.312630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:44.339020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b14ae-cb93-44a7-9ce5-523ba2c063b6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.380637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:44.414722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f028a8a-bbf6-4a22-b752-99e8b06c865f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.422109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e9013-36db-4406-b634-efa317dafe03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.423427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e9013-36db-4406-b634-efa317dafe03 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.423867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e9013-36db-4406-b634-efa317dafe03 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.424157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e9013-36db-4406-b634-efa317dafe03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.426356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1b5e9013-36db-4406-b634-efa317dafe03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:44.439655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd73b30-c05e-403f-bcd3-204bc2f3a4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.441402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd73b30-c05e-403f-bcd3-204bc2f3a4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.441803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd73b30-c05e-403f-bcd3-204bc2f3a4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.442189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd73b30-c05e-403f-bcd3-204bc2f3a4f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.443055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd73b30-c05e-403f-bcd3-204bc2f3a4f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:44.443705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd73b30-c05e-403f-bcd3-204bc2f3a4f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.445738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd73b30-c05e-403f-bcd3-204bc2f3a4f3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.448297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e9013-36db-4406-b634-efa317dafe03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.450327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e9013-36db-4406-b634-efa317dafe03 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.463379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 898/1816] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:44.470023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd73b30-c05e-403f-bcd3-204bc2f3a4f3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.473137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 919/1817] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:44.473996 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e9013-36db-4406-b634-efa317dafe03 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.490843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e9013-36db-4406-b634-efa317dafe03 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.498980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd73b30-c05e-403f-bcd3-204bc2f3a4f3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.500275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3df558-3113-4ed8-92fe-34c7cc56e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.500972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3df558-3113-4ed8-92fe-34c7cc56e7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.502100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3df558-3113-4ed8-92fe-34c7cc56e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.502805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3df558-3113-4ed8-92fe-34c7cc56e7b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.503321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3df558-3113-4ed8-92fe-34c7cc56e7b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.506776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3df558-3113-4ed8-92fe-34c7cc56e7b0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.513964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbdb27-7232-4694-b03b-503c96659f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.515639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbdb27-7232-4694-b03b-503c96659f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.517495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbdb27-7232-4694-b03b-503c96659f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.517903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbdb27-7232-4694-b03b-503c96659f81 admin admin] PATH_INFO: `/v3/projects/ac17974f6fd848389abe1553e561be8d/users/0d64da65cf9b45d5a3233414538d0b43/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.556293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbdb27-7232-4694-b03b-503c96659f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac17974f6fd848389abe1553e561be8d, user_id=0d64da65cf9b45d5a3233414538d0b43, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.559892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbdb27-7232-4694-b03b-503c96659f81 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.572298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 920/1818] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:44 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:44.582020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3df558-3113-4ed8-92fe-34c7cc56e7b0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.586504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 899/1819] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:44 2022] PUT /identity/v3/projects/ac17974f6fd848389abe1553e561be8d/users/0d64da65cf9b45d5a3233414538d0b43/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:44.589452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbdb27-7232-4694-b03b-503c96659f81 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.691130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbdb27-7232-4694-b03b-503c96659f81 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.695980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3df558-3113-4ed8-92fe-34c7cc56e7b0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.697386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9945a51-7b03-4666-b689-9665fb67beeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.698206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9945a51-7b03-4666-b689-9665fb67beeb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.698596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9945a51-7b03-4666-b689-9665fb67beeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.698976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9945a51-7b03-4666-b689-9665fb67beeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.699876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9945a51-7b03-4666-b689-9665fb67beeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.701619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9945a51-7b03-4666-b689-9665fb67beeb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.705488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c99fb-7234-4059-a760-285a24b50462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.706556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c99fb-7234-4059-a760-285a24b50462 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.707706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c99fb-7234-4059-a760-285a24b50462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.708179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c99fb-7234-4059-a760-285a24b50462 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.709428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c99fb-7234-4059-a760-285a24b50462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.714054 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 900/1820] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:44.714279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c99fb-7234-4059-a760-285a24b50462 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.719980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9945a51-7b03-4666-b689-9665fb67beeb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.731606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9945a51-7b03-4666-b689-9665fb67beeb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.738637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef82a75-59ea-43fc-8c0c-deb391633c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.740011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef82a75-59ea-43fc-8c0c-deb391633c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.740301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef82a75-59ea-43fc-8c0c-deb391633c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.740585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef82a75-59ea-43fc-8c0c-deb391633c74 admin admin] PATH_INFO: `/v3/projects/ac17974f6fd848389abe1553e561be8d/users/0d64da65cf9b45d5a3233414538d0b43/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.743638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef82a75-59ea-43fc-8c0c-deb391633c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac17974f6fd848389abe1553e561be8d, user_id=0d64da65cf9b45d5a3233414538d0b43, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.747178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef82a75-59ea-43fc-8c0c-deb391633c74 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.757700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 921/1821] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:44 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:44.762708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c99fb-7234-4059-a760-285a24b50462 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.769213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 901/1822] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:44 2022] PUT /identity/v3/projects/ac17974f6fd848389abe1553e561be8d/users/0d64da65cf9b45d5a3233414538d0b43/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:44.774517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c99fb-7234-4059-a760-285a24b50462 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.785422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9051af-4832-469d-8217-b72c228629a4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.785835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9051af-4832-469d-8217-b72c228629a4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.786221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9051af-4832-469d-8217-b72c228629a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.833839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-de9051af-4832-469d-8217-b72c228629a4 None None] MFA Rules not processed for user `0d64da65cf9b45d5a3233414538d0b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:44.835736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-de9051af-4832-469d-8217-b72c228629a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:44.861573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f711647-0e31-4d17-b7ef-1afd849bb353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.862500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f711647-0e31-4d17-b7ef-1afd849bb353 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.862931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f711647-0e31-4d17-b7ef-1afd849bb353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.863867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f711647-0e31-4d17-b7ef-1afd849bb353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.864851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f711647-0e31-4d17-b7ef-1afd849bb353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.866592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f711647-0e31-4d17-b7ef-1afd849bb353 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.881169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 922/1823] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:44.882076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f711647-0e31-4d17-b7ef-1afd849bb353 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.888101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1f711647-0e31-4d17-b7ef-1afd849bb353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:44.898383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:44.907850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f711647-0e31-4d17-b7ef-1afd849bb353 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:44.917691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:44.918591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:44.918880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:44.919169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] PATH_INFO: `/v3/projects/1aebb6b957bb4988b1e6dfdb335bbb89` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:44.920702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aebb6b957bb4988b1e6dfdb335bbb89)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:44.922659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:44.965335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 902/1824] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:44.966819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9051af-4832-469d-8217-b72c228629a4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:44.977688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:44.986947 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9051af-4832-469d-8217-b72c228629a4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:45.013332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aebb6b957bb4988b1e6dfdb335bbb89', 'request_id': 'req-26383d3e-faa7-4930-afac-1874534f99c4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:45.029320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aebb6b957bb4988b1e6dfdb335bbb89', 'request_id': 'req-26383d3e-faa7-4930-afac-1874534f99c4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:45.038082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aebb6b957bb4988b1e6dfdb335bbb89', 'request_id': 'req-26383d3e-faa7-4930-afac-1874534f99c4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:45.056688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] The token cache is being invalidate because project 1aebb6b957bb4988b1e6dfdb335bbb89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:45.058514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:45.059362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:45.061096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 923/1825] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:44 2022] DELETE /identity/v3/projects/1aebb6b957bb4988b1e6dfdb335bbb89 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:45.062042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:45.065880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:45.074812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77611ca1-a237-4dd8-ae12-41ddd9745bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:45.076046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611ca1-a237-4dd8-ae12-41ddd9745bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.076489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611ca1-a237-4dd8-ae12-41ddd9745bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.076943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611ca1-a237-4dd8-ae12-41ddd9745bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.078061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77611ca1-a237-4dd8-ae12-41ddd9745bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:45.080336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77611ca1-a237-4dd8-ae12-41ddd9745bb6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:45.095455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 903/1826] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:45.095455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada97b0-235a-4b8a-9976-75956ee3a74e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.095455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada97b0-235a-4b8a-9976-75956ee3a74e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.095455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada97b0-235a-4b8a-9976-75956ee3a74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.126653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3ada97b0-235a-4b8a-9976-75956ee3a74e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:45.130185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ada97b0-235a-4b8a-9976-75956ee3a74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:45.151047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 904/1827] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:45.151047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada97b0-235a-4b8a-9976-75956ee3a74e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:45.152993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ada97b0-235a-4b8a-9976-75956ee3a74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:45.174805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada97b0-235a-4b8a-9976-75956ee3a74e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:45.181321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25074f90-ff40-40d0-923c-dcc9ca41fe1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:45.182698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25074f90-ff40-40d0-923c-dcc9ca41fe1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.183199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25074f90-ff40-40d0-923c-dcc9ca41fe1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.183722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25074f90-ff40-40d0-923c-dcc9ca41fe1a admin admin] PATH_INFO: `/v3/projects/e7a3630a87d44b56810bfa23ecc92996/users/7218a1a982904fdf845b8866bed3a8b8/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.207466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25074f90-ff40-40d0-923c-dcc9ca41fe1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a3630a87d44b56810bfa23ecc92996, user_id=7218a1a982904fdf845b8866bed3a8b8, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:45.212694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25074f90-ff40-40d0-923c-dcc9ca41fe1a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:45.222684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26383d3e-faa7-4930-afac-1874534f99c4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:45.240411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403be3d-4f99-47aa-ad2d-36972380a3ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:45.241525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403be3d-4f99-47aa-ad2d-36972380a3ee service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.243492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403be3d-4f99-47aa-ad2d-36972380a3ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.244055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403be3d-4f99-47aa-ad2d-36972380a3ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.248787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a403be3d-4f99-47aa-ad2d-36972380a3ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:45.286217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 905/1828] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:45 2022] PUT /identity/v3/projects/e7a3630a87d44b56810bfa23ecc92996/users/7218a1a982904fdf845b8866bed3a8b8/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:45.287850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45414c-93bb-4de0-b828-9043934be14b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.292832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45414c-93bb-4de0-b828-9043934be14b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.293669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45414c-93bb-4de0-b828-9043934be14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.341974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6a45414c-93bb-4de0-b828-9043934be14b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:45.358712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6a45414c-93bb-4de0-b828-9043934be14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:45.481054 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 906/1829] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:45.484495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403be3d-4f99-47aa-ad2d-36972380a3ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:45.486191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18f1b9-8bdb-46e2-a83b-719d2d07c013 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.486490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18f1b9-8bdb-46e2-a83b-719d2d07c013 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.486936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18f1b9-8bdb-46e2-a83b-719d2d07c013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.488054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403be3d-4f99-47aa-ad2d-36972380a3ee service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:45.520701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fe18f1b9-8bdb-46e2-a83b-719d2d07c013 None None] MFA Rules not processed for user `7218a1a982904fdf845b8866bed3a8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:45.521980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fe18f1b9-8bdb-46e2-a83b-719d2d07c013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:45.522933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 924/1830] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:45.525277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403be3d-4f99-47aa-ad2d-36972380a3ee service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:45.555943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403be3d-4f99-47aa-ad2d-36972380a3ee service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:45.572008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4816b-3684-4e13-933e-5fcc58bb4e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:45.572693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4816b-3684-4e13-933e-5fcc58bb4e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.573028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4816b-3684-4e13-933e-5fcc58bb4e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.573312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4816b-3684-4e13-933e-5fcc58bb4e63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.574064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4816b-3684-4e13-933e-5fcc58bb4e63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:45.574403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4816b-3684-4e13-933e-5fcc58bb4e63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:45.581040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4816b-3684-4e13-933e-5fcc58bb4e63 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:45.614985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 925/1831] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:45.616689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4816b-3684-4e13-933e-5fcc58bb4e63 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:45.643119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4816b-3684-4e13-933e-5fcc58bb4e63 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:45.658111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd5510-d6bb-4d1f-bdd8-26a3cd947bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:45.658894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd5510-d6bb-4d1f-bdd8-26a3cd947bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.659185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd5510-d6bb-4d1f-bdd8-26a3cd947bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.659554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd5510-d6bb-4d1f-bdd8-26a3cd947bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.660377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd5510-d6bb-4d1f-bdd8-26a3cd947bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:45.663421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd5510-d6bb-4d1f-bdd8-26a3cd947bd5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:45.705275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:45.723903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 907/1832] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:45 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:45.727565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18f1b9-8bdb-46e2-a83b-719d2d07c013 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:45.755998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 926/1833] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:45 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:14:45.757336 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd5510-d6bb-4d1f-bdd8-26a3cd947bd5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:45.761557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18f1b9-8bdb-46e2-a83b-719d2d07c013 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:45.768459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:45.885609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd5510-d6bb-4d1f-bdd8-26a3cd947bd5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:45.905577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dcff1-95e8-4ab0-9829-07820b876fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:45.911502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dcff1-95e8-4ab0-9829-07820b876fad admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.911861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dcff1-95e8-4ab0-9829-07820b876fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.912132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dcff1-95e8-4ab0-9829-07820b876fad admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.914183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dcff1-95e8-4ab0-9829-07820b876fad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:45.917584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d251ae-d984-4c88-be73-75b3aa943ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:45.918544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d251ae-d984-4c88-be73-75b3aa943ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:45.918830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d251ae-d984-4c88-be73-75b3aa943ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:45.919117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d251ae-d984-4c88-be73-75b3aa943ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:45.919882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d251ae-d984-4c88-be73-75b3aa943ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:45.921894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d251ae-d984-4c88-be73-75b3aa943ab8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:45.922112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dcff1-95e8-4ab0-9829-07820b876fad admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:45.980858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 908/1834] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:45 2022] POST /identity/v3/projects => generated 365 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:45.989910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d251ae-d984-4c88-be73-75b3aa943ab8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.013063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 927/1835] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:45 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:14:46.017867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dcff1-95e8-4ab0-9829-07820b876fad admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.060232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dcff1-95e8-4ab0-9829-07820b876fad admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.105932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d251ae-d984-4c88-be73-75b3aa943ab8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.122039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f316f4-322e-49c8-bdf2-ae857676bd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.122429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f316f4-322e-49c8-bdf2-ae857676bd33 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.122723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f316f4-322e-49c8-bdf2-ae857676bd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.123084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f316f4-322e-49c8-bdf2-ae857676bd33 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.123922 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f316f4-322e-49c8-bdf2-ae857676bd33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.126439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f316f4-322e-49c8-bdf2-ae857676bd33 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.165442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 909/1836] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:45 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:46.167084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f316f4-322e-49c8-bdf2-ae857676bd33 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.185109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f316f4-322e-49c8-bdf2-ae857676bd33 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.193772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dc648-7335-411a-8548-3c0ce982c6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.194560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dc648-7335-411a-8548-3c0ce982c6bf admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.194897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dc648-7335-411a-8548-3c0ce982c6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.195272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dc648-7335-411a-8548-3c0ce982c6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.196115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dc648-7335-411a-8548-3c0ce982c6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.198784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dc648-7335-411a-8548-3c0ce982c6bf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.212861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 910/1837] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:46.217649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dc648-7335-411a-8548-3c0ce982c6bf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.247290 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b458949-6733-4a8b-8419-d46eed9a78fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.253132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b458949-6733-4a8b-8419-d46eed9a78fb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.253512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b458949-6733-4a8b-8419-d46eed9a78fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.253908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b458949-6733-4a8b-8419-d46eed9a78fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.254719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b458949-6733-4a8b-8419-d46eed9a78fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.256830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b458949-6733-4a8b-8419-d46eed9a78fb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.272964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dc648-7335-411a-8548-3c0ce982c6bf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.275908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 928/1838] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:46.277994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b458949-6733-4a8b-8419-d46eed9a78fb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.282036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130e73-bda6-414d-95e7-f9cc9c0871a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.283116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130e73-bda6-414d-95e7-f9cc9c0871a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.283600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130e73-bda6-414d-95e7-f9cc9c0871a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.283989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130e73-bda6-414d-95e7-f9cc9c0871a6 admin admin] PATH_INFO: `/v3/projects/eae4242ed93445e2821db5f5081e9459/users/02ce84d5002c4da6aeec37369c11324a/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.291563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b458949-6733-4a8b-8419-d46eed9a78fb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.299173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564da55-1d52-4c4f-9571-3ad1632a8fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.300451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564da55-1d52-4c4f-9571-3ad1632a8fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.302025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564da55-1d52-4c4f-9571-3ad1632a8fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.302025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564da55-1d52-4c4f-9571-3ad1632a8fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.302566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564da55-1d52-4c4f-9571-3ad1632a8fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.305127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e130e73-bda6-414d-95e7-f9cc9c0871a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae4242ed93445e2821db5f5081e9459, user_id=02ce84d5002c4da6aeec37369c11324a, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.305270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564da55-1d52-4c4f-9571-3ad1632a8fd5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.310442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e130e73-bda6-414d-95e7-f9cc9c0871a6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.321355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 929/1839] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:46.336180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564da55-1d52-4c4f-9571-3ad1632a8fd5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.348572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 911/1840] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:46 2022] PUT /identity/v3/projects/eae4242ed93445e2821db5f5081e9459/users/02ce84d5002c4da6aeec37369c11324a/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:46.352316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130e73-bda6-414d-95e7-f9cc9c0871a6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.357507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564da55-1d52-4c4f-9571-3ad1632a8fd5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.407613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:46.433004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130e73-bda6-414d-95e7-f9cc9c0871a6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.435627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f13b3-9900-4d27-aef8-f2c7ee1a3e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.436597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f13b3-9900-4d27-aef8-f2c7ee1a3e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.437084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f13b3-9900-4d27-aef8-f2c7ee1a3e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.437186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f13b3-9900-4d27-aef8-f2c7ee1a3e86 admin admin] PATH_INFO: `/v3/projects/db815250738b49779631fbdf99f861e2/users/999ee31e50b9434484f3586a9db69635/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.449851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc325e-8972-47d6-8995-a359d74c4e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.450622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc325e-8972-47d6-8995-a359d74c4e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.451052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc325e-8972-47d6-8995-a359d74c4e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.451422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc325e-8972-47d6-8995-a359d74c4e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.452160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc325e-8972-47d6-8995-a359d74c4e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.453975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc325e-8972-47d6-8995-a359d74c4e2c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.466496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f13b3-9900-4d27-aef8-f2c7ee1a3e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db815250738b49779631fbdf99f861e2, user_id=999ee31e50b9434484f3586a9db69635, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.467107 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 912/1841] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:46.469533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f13b3-9900-4d27-aef8-f2c7ee1a3e86 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.471173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc325e-8972-47d6-8995-a359d74c4e2c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.486241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc325e-8972-47d6-8995-a359d74c4e2c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.487723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 930/1842] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:46 2022] PUT /identity/v3/projects/db815250738b49779631fbdf99f861e2/users/999ee31e50b9434484f3586a9db69635/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:46.498377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0185d4-512a-4646-845b-f9e97035c3f0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.498674 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0185d4-512a-4646-845b-f9e97035c3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.498954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0185d4-512a-4646-845b-f9e97035c3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.506923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:46.533864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2a0185d4-512a-4646-845b-f9e97035c3f0 None None] MFA Rules not processed for user `999ee31e50b9434484f3586a9db69635`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:46.535171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2a0185d4-512a-4646-845b-f9e97035c3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:46.561891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb1fa1-32df-4fe8-b926-44f8e6b87b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.562832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb1fa1-32df-4fe8-b926-44f8e6b87b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.563117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb1fa1-32df-4fe8-b926-44f8e6b87b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.564338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb1fa1-32df-4fe8-b926-44f8e6b87b35 admin admin] PATH_INFO: `/v3/projects/eae4242ed93445e2821db5f5081e9459/users/02ce84d5002c4da6aeec37369c11324a/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.566034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb1fa1-32df-4fe8-b926-44f8e6b87b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae4242ed93445e2821db5f5081e9459, user_id=02ce84d5002c4da6aeec37369c11324a, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.568923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb1fa1-32df-4fe8-b926-44f8e6b87b35 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.586421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 913/1843] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:46 2022] PUT /identity/v3/projects/eae4242ed93445e2821db5f5081e9459/users/02ce84d5002c4da6aeec37369c11324a/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:46.610016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f3d98-b01c-4ca4-a503-9b171da783a7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.610821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f3d98-b01c-4ca4-a503-9b171da783a7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.611104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f3d98-b01c-4ca4-a503-9b171da783a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.630762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 931/1844] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:46.631845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0185d4-512a-4646-845b-f9e97035c3f0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.641808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-129f3d98-b01c-4ca4-a503-9b171da783a7 None None] MFA Rules not processed for user `02ce84d5002c4da6aeec37369c11324a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:46.642965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0185d4-512a-4646-845b-f9e97035c3f0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.643078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-129f3d98-b01c-4ca4-a503-9b171da783a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:46.700958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f59a4d-acfe-430f-8a7f-de82f02aee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.701793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f59a4d-acfe-430f-8a7f-de82f02aee41 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.702083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f59a4d-acfe-430f-8a7f-de82f02aee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.702364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f59a4d-acfe-430f-8a7f-de82f02aee41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.703128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f59a4d-acfe-430f-8a7f-de82f02aee41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.705143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f59a4d-acfe-430f-8a7f-de82f02aee41 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.714226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 914/1845] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:46.726747 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f3d98-b01c-4ca4-a503-9b171da783a7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.732274 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 932/1846] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:46 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:46.736647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f59a4d-acfe-430f-8a7f-de82f02aee41 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.788247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f3d98-b01c-4ca4-a503-9b171da783a7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.799339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:46.803686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f59a4d-acfe-430f-8a7f-de82f02aee41 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.809091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742aa95d-ab13-4934-a171-14f40e2b3396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.809780 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aa95d-ab13-4934-a171-14f40e2b3396 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.810068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aa95d-ab13-4934-a171-14f40e2b3396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.810349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aa95d-ab13-4934-a171-14f40e2b3396 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.811057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742aa95d-ab13-4934-a171-14f40e2b3396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.813034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742aa95d-ab13-4934-a171-14f40e2b3396 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.846438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 933/1847] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:46 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:14:46.850640 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742aa95d-ab13-4934-a171-14f40e2b3396 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.853409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3b32-0f2c-42fc-8e71-f23aef83f496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.854182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3b32-0f2c-42fc-8e71-f23aef83f496 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.854545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3b32-0f2c-42fc-8e71-f23aef83f496 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.854893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3b32-0f2c-42fc-8e71-f23aef83f496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.862517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742aa95d-ab13-4934-a171-14f40e2b3396 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.867644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6732872-dc61-4e1f-be32-f9fb1db1e4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.868317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6732872-dc61-4e1f-be32-f9fb1db1e4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.868606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6732872-dc61-4e1f-be32-f9fb1db1e4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.868886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6732872-dc61-4e1f-be32-f9fb1db1e4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.869652 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6732872-dc61-4e1f-be32-f9fb1db1e4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.871282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6732872-dc61-4e1f-be32-f9fb1db1e4f1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.872838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:46.884406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 934/1848] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:46.888266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6732872-dc61-4e1f-be32-f9fb1db1e4f1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.898913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6732872-dc61-4e1f-be32-f9fb1db1e4f1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:46.904417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ace346-a2d5-4518-8087-e7a1a1c09a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:46.905490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ace346-a2d5-4518-8087-e7a1a1c09a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:46.905783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ace346-a2d5-4518-8087-e7a1a1c09a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:46.906165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ace346-a2d5-4518-8087-e7a1a1c09a2e admin admin] PATH_INFO: `/v3/projects/52d7fed635f74eeba01057ef0a151d24/users/f756770abf6845fcaf783eac11b20310/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:46.925413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ace346-a2d5-4518-8087-e7a1a1c09a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d7fed635f74eeba01057ef0a151d24, user_id=f756770abf6845fcaf783eac11b20310, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.927210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c3b32-0f2c-42fc-8e71-f23aef83f496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:46.928240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ace346-a2d5-4518-8087-e7a1a1c09a2e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.928830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c3b32-0f2c-42fc-8e71-f23aef83f496 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:46.940644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 915/1849] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:46.942408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 935/1850] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:46 2022] PUT /identity/v3/projects/52d7fed635f74eeba01057ef0a151d24/users/f756770abf6845fcaf783eac11b20310/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:46.946397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3b32-0f2c-42fc-8e71-f23aef83f496 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:46.959225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3b32-0f2c-42fc-8e71-f23aef83f496 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:47.021818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301519a2-85fc-47b9-8095-ee4d0819c4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:47.022504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301519a2-85fc-47b9-8095-ee4d0819c4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.022786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301519a2-85fc-47b9-8095-ee4d0819c4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.023064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301519a2-85fc-47b9-8095-ee4d0819c4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.023843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301519a2-85fc-47b9-8095-ee4d0819c4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:47.025440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301519a2-85fc-47b9-8095-ee4d0819c4c5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:47.037802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 916/1851] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:47.041834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ace346-a2d5-4518-8087-e7a1a1c09a2e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:47.057640 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ace346-a2d5-4518-8087-e7a1a1c09a2e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:47.069115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7eed-3d4d-41e4-a3af-5c406d1684bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:47.070355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7eed-3d4d-41e4-a3af-5c406d1684bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.070808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7eed-3d4d-41e4-a3af-5c406d1684bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.071191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7eed-3d4d-41e4-a3af-5c406d1684bf admin admin] PATH_INFO: `/v3/projects/52d7fed635f74eeba01057ef0a151d24/users/f756770abf6845fcaf783eac11b20310/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.073816 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a7eed-3d4d-41e4-a3af-5c406d1684bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d7fed635f74eeba01057ef0a151d24, user_id=f756770abf6845fcaf783eac11b20310, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:47.076974 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a7eed-3d4d-41e4-a3af-5c406d1684bf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:47.090015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 936/1852] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:47 2022] PUT /identity/v3/projects/52d7fed635f74eeba01057ef0a151d24/users/f756770abf6845fcaf783eac11b20310/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:47.103917 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca86ffe-a38a-4b31-9ee1-ef482d89f776 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.104210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca86ffe-a38a-4b31-9ee1-ef482d89f776 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.104488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca86ffe-a38a-4b31-9ee1-ef482d89f776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.138684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-aca86ffe-a38a-4b31-9ee1-ef482d89f776 None None] MFA Rules not processed for user `f756770abf6845fcaf783eac11b20310`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:47.139782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-aca86ffe-a38a-4b31-9ee1-ef482d89f776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:47.192356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:47.233345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 917/1853] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:47.244421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7eed-3d4d-41e4-a3af-5c406d1684bf admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:47.246099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-671a7eed-3d4d-41e4-a3af-5c406d1684bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:47.309145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7eed-3d4d-41e4-a3af-5c406d1684bf admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:47.313243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555edddb-7962-4cd5-b0f1-56fa208377eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:47.313956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555edddb-7962-4cd5-b0f1-56fa208377eb service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.314242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555edddb-7962-4cd5-b0f1-56fa208377eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.314519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555edddb-7962-4cd5-b0f1-56fa208377eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.324345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555edddb-7962-4cd5-b0f1-56fa208377eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:47.325785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555edddb-7962-4cd5-b0f1-56fa208377eb service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:47.337357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 937/1854] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:47.450929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca86ffe-a38a-4b31-9ee1-ef482d89f776 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:47.517560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555edddb-7962-4cd5-b0f1-56fa208377eb service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:47.559807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca86ffe-a38a-4b31-9ee1-ef482d89f776 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:47.565543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:47.566535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.566885 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.567171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] PATH_INFO: `/v3/users/7218a1a982904fdf845b8866bed3a8b8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.568484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=7218a1a982904fdf845b8866bed3a8b8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:47.570354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:47.605993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555edddb-7962-4cd5-b0f1-56fa208377eb service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:47.611499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:47.612852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.613141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.613465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] PATH_INFO: `/v3/users/999ee31e50b9434484f3586a9db69635` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.615215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] RBAC: Authorizing `identity:delete_user(user_id=999ee31e50b9434484f3586a9db69635)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:47.617653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:47.647199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7218a1a982904fdf845b8866bed3a8b8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.659793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7218a1a982904fdf845b8866bed3a8b8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.670698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7218a1a982904fdf845b8866bed3a8b8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.686586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7218a1a982904fdf845b8866bed3a8b8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.687482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-22e3c6a4-5d85-473a-8d29-a4a39fc49bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7218a1a982904fdf845b8866bed3a8b8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.688125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999ee31e50b9434484f3586a9db69635'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.689281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999ee31e50b9434484f3586a9db69635'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.695442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 918/1855] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:47 2022] DELETE /identity/v3/users/7218a1a982904fdf845b8866bed3a8b8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:47.705504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e7131-ac63-45ba-81d4-c87ae9bd2752 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.705761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999ee31e50b9434484f3586a9db69635'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.706216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e7131-ac63-45ba-81d4-c87ae9bd2752 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.706801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e7131-ac63-45ba-81d4-c87ae9bd2752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.717686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:47.724393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999ee31e50b9434484f3586a9db69635'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.734343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dc5e7131-ac63-45ba-81d4-c87ae9bd2752 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:47.735326 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b34abc37-06ac-4764-807f-5e35155becec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999ee31e50b9434484f3586a9db69635'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:47.735904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dc5e7131-ac63-45ba-81d4-c87ae9bd2752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:47.742600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 938/1856] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:47 2022] DELETE /identity/v3/users/999ee31e50b9434484f3586a9db69635 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:47.747552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583bc54-bd63-45d3-8404-be99619d6b14 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.747927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583bc54-bd63-45d3-8404-be99619d6b14 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.748342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583bc54-bd63-45d3-8404-be99619d6b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.771398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d583bc54-bd63-45d3-8404-be99619d6b14 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:47.772644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d583bc54-bd63-45d3-8404-be99619d6b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:47.812942 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 919/1857] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:47.824617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e7131-ac63-45ba-81d4-c87ae9bd2752 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:47.826063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dc5e7131-ac63-45ba-81d4-c87ae9bd2752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:47.833834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:47.841626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 939/1858] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:47.852173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583bc54-bd63-45d3-8404-be99619d6b14 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:47.853576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d583bc54-bd63-45d3-8404-be99619d6b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:47.897968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e7131-ac63-45ba-81d4-c87ae9bd2752 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:47.917789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583bc54-bd63-45d3-8404-be99619d6b14 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:47.921445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db381a-027b-466f-bea1-d6462c981683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:47.922103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db381a-027b-466f-bea1-d6462c981683 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.922392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db381a-027b-466f-bea1-d6462c981683 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.922671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db381a-027b-466f-bea1-d6462c981683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.933094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db381a-027b-466f-bea1-d6462c981683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:47.934552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db381a-027b-466f-bea1-d6462c981683 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:47.945524 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 940/1859] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:47.950245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e8357-c5cc-4ef2-b92d-c8d71b29d5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:47.950943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e8357-c5cc-4ef2-b92d-c8d71b29d5ca service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:47.951227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e8357-c5cc-4ef2-b92d-c8d71b29d5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:47.951531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e8357-c5cc-4ef2-b92d-c8d71b29d5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:47.962022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e8357-c5cc-4ef2-b92d-c8d71b29d5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:47.963506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e8357-c5cc-4ef2-b92d-c8d71b29d5ca service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:47.973751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 920/1860] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:48.317587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db381a-027b-466f-bea1-d6462c981683 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:48.319092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b6db381a-027b-466f-bea1-d6462c981683 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:48.338239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db381a-027b-466f-bea1-d6462c981683 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:48.345422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:48.353475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:48.353778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:48.354063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] PATH_INFO: `/v3/projects/db815250738b49779631fbdf99f861e2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:48.355538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db815250738b49779631fbdf99f861e2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:48.357483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:48.415742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db815250738b49779631fbdf99f861e2', 'request_id': 'req-e734e1c3-a6d6-4780-9022-3d7f606d373e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.423560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db815250738b49779631fbdf99f861e2', 'request_id': 'req-e734e1c3-a6d6-4780-9022-3d7f606d373e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.427035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e8357-c5cc-4ef2-b92d-c8d71b29d5ca service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:48.428475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-923e8357-c5cc-4ef2-b92d-c8d71b29d5ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:48.433572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db815250738b49779631fbdf99f861e2', 'request_id': 'req-e734e1c3-a6d6-4780-9022-3d7f606d373e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.447392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e8357-c5cc-4ef2-b92d-c8d71b29d5ca service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:48.458527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] The token cache is being invalidate because project db815250738b49779631fbdf99f861e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:48.458848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.459526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.461706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 941/1861] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:48 2022] DELETE /identity/v3/projects/db815250738b49779631fbdf99f861e2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:48.473351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:48.474896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:48.492381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:48.537522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:48.538476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:48.538860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:48.539160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] PATH_INFO: `/v3/projects/e7a3630a87d44b56810bfa23ecc92996` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:48.540530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a3630a87d44b56810bfa23ecc92996)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:48.542473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:48.553902 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734e1c3-a6d6-4780-9022-3d7f606d373e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:48.560514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:48.561402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:48.561692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:48.562125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] PATH_INFO: `/v3/users/f756770abf6845fcaf783eac11b20310` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:48.563209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f756770abf6845fcaf783eac11b20310)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:48.565302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:48.610456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a3630a87d44b56810bfa23ecc92996', 'request_id': 'req-67dc9662-c619-49f4-9b69-d923804daf3a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.622247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a3630a87d44b56810bfa23ecc92996', 'request_id': 'req-67dc9662-c619-49f4-9b69-d923804daf3a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.634323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a3630a87d44b56810bfa23ecc92996', 'request_id': 'req-67dc9662-c619-49f4-9b69-d923804daf3a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.636989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:48.651399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f756770abf6845fcaf783eac11b20310'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.652245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f756770abf6845fcaf783eac11b20310'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.664647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] The token cache is being invalidate because project e7a3630a87d44b56810bfa23ecc92996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:48.664946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.666410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.675382 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 921/1862] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:48 2022] DELETE /identity/v3/projects/e7a3630a87d44b56810bfa23ecc92996 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:48.676493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:48.678324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:48.680681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f756770abf6845fcaf783eac11b20310'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.718321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f756770abf6845fcaf783eac11b20310'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.748157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f756770abf6845fcaf783eac11b20310'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:48.762623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 942/1863] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:48 2022] DELETE /identity/v3/users/f756770abf6845fcaf783eac11b20310 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:48.807511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc9662-c619-49f4-9b69-d923804daf3a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:48.956016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:48.956585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:48.957218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:48.957329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] PATH_INFO: `/v3/users/02ce84d5002c4da6aeec37369c11324a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:48.959484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ce84d5002c4da6aeec37369c11324a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:48.961643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:49.052199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ce84d5002c4da6aeec37369c11324a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.061503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ce84d5002c4da6aeec37369c11324a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.071408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ce84d5002c4da6aeec37369c11324a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.092646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ce84d5002c4da6aeec37369c11324a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.093529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ce84d5002c4da6aeec37369c11324a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.102960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 922/1864] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:48 2022] DELETE /identity/v3/users/02ce84d5002c4da6aeec37369c11324a => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:49.382174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:49.384977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:49.400456 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9975-d772-45c3-851d-6bb3c183521d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:49.465246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:49.466359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.466762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.467235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] PATH_INFO: `/v3/projects/52d7fed635f74eeba01057ef0a151d24` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.468633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d7fed635f74eeba01057ef0a151d24)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:49.470876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:49.494683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:49.496908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:49.513904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1839722-5177-4fa0-aa50-6ff705ef70f2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:49.519242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:49.520558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.521012 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.521182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] PATH_INFO: `/v3/projects/eae4242ed93445e2821db5f5081e9459` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.522464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae4242ed93445e2821db5f5081e9459)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:49.524928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:49.529070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:49.538707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:49.557259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d7fed635f74eeba01057ef0a151d24', 'request_id': 'req-5120af8c-fd72-485a-ad2f-8eb035f0beb9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.566372 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d7fed635f74eeba01057ef0a151d24', 'request_id': 'req-5120af8c-fd72-485a-ad2f-8eb035f0beb9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.584773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d7fed635f74eeba01057ef0a151d24', 'request_id': 'req-5120af8c-fd72-485a-ad2f-8eb035f0beb9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.598537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae4242ed93445e2821db5f5081e9459', 'request_id': 'req-424321ad-3377-4179-852f-768ae7aa69ee'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.614586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae4242ed93445e2821db5f5081e9459', 'request_id': 'req-424321ad-3377-4179-852f-768ae7aa69ee'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.618863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] The token cache is being invalidate because project 52d7fed635f74eeba01057ef0a151d24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:49.620467 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.621382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5120af8c-fd72-485a-ad2f-8eb035f0beb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.623735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 943/1865] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:49 2022] DELETE /identity/v3/projects/52d7fed635f74eeba01057ef0a151d24 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:49.627627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae4242ed93445e2821db5f5081e9459', 'request_id': 'req-424321ad-3377-4179-852f-768ae7aa69ee'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.635778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bb4d4-e379-4aef-aebd-be8c2733c5fd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.636108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bb4d4-e379-4aef-aebd-be8c2733c5fd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.636398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bb4d4-e379-4aef-aebd-be8c2733c5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.641601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] The token cache is being invalidate because project eae4242ed93445e2821db5f5081e9459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:49.642227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.643025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-424321ad-3377-4179-852f-768ae7aa69ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:49.644970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 923/1866] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:49 2022] DELETE /identity/v3/projects/eae4242ed93445e2821db5f5081e9459 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:49.656303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02740261-b825-4a59-b97e-41c9795368ce None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.656303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02740261-b825-4a59-b97e-41c9795368ce None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.656303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02740261-b825-4a59-b97e-41c9795368ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.664011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-531bb4d4-e379-4aef-aebd-be8c2733c5fd None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:49.665380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-531bb4d4-e379-4aef-aebd-be8c2733c5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:49.678861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-02740261-b825-4a59-b97e-41c9795368ce None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:49.680971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-02740261-b825-4a59-b97e-41c9795368ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:49.747576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 944/1867] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:49.755888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427af1c-67c0-4ce9-8b1f-dfc9642eae82 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.756188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427af1c-67c0-4ce9-8b1f-dfc9642eae82 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.756602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427af1c-67c0-4ce9-8b1f-dfc9642eae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.766410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 924/1868] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:49.774103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bbed4-7e57-44ca-9990-caa48ae70d13 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.774405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bbed4-7e57-44ca-9990-caa48ae70d13 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.774762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bbed4-7e57-44ca-9990-caa48ae70d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.777236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2427af1c-67c0-4ce9-8b1f-dfc9642eae82 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:49.778436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2427af1c-67c0-4ce9-8b1f-dfc9642eae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:49.790484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 945/1869] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:49.794637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427af1c-67c0-4ce9-8b1f-dfc9642eae82 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:49.802016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6a7bbed4-7e57-44ca-9990-caa48ae70d13 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:49.804614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6a7bbed4-7e57-44ca-9990-caa48ae70d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:49.804778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427af1c-67c0-4ce9-8b1f-dfc9642eae82 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:49.809969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57688240-8ca4-4142-ab88-389636d4bdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:49.810723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57688240-8ca4-4142-ab88-389636d4bdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.811018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57688240-8ca4-4142-ab88-389636d4bdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.811347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57688240-8ca4-4142-ab88-389636d4bdc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.812117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57688240-8ca4-4142-ab88-389636d4bdc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:49.812383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57688240-8ca4-4142-ab88-389636d4bdc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:49.813873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 925/1870] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:49.814192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57688240-8ca4-4142-ab88-389636d4bdc9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:49.818033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bbed4-7e57-44ca-9990-caa48ae70d13 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:49.827324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 946/1871] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:14:49.831506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bbed4-7e57-44ca-9990-caa48ae70d13 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:49.831659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57688240-8ca4-4142-ab88-389636d4bdc9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:49.836503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36bcea-a2bb-4029-87e0-84ef75c39a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:49.837268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36bcea-a2bb-4029-87e0-84ef75c39a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.837625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36bcea-a2bb-4029-87e0-84ef75c39a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.837972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36bcea-a2bb-4029-87e0-84ef75c39a24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.838783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36bcea-a2bb-4029-87e0-84ef75c39a24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:49.839156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36bcea-a2bb-4029-87e0-84ef75c39a24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:49.840883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36bcea-a2bb-4029-87e0-84ef75c39a24 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:49.842289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57688240-8ca4-4142-ab88-389636d4bdc9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:49.847918 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f72f6-a826-49c7-86a1-0c7a9b7f4b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:49.848586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f72f6-a826-49c7-86a1-0c7a9b7f4b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.848880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f72f6-a826-49c7-86a1-0c7a9b7f4b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.849161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f72f6-a826-49c7-86a1-0c7a9b7f4b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.849884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f72f6-a826-49c7-86a1-0c7a9b7f4b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:49.851868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f72f6-a826-49c7-86a1-0c7a9b7f4b74 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:49.855081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 926/1872] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:14:49.859476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36bcea-a2bb-4029-87e0-84ef75c39a24 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:49.870605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36bcea-a2bb-4029-87e0-84ef75c39a24 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:49.876092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64671dc-8694-4887-9df7-4150d7f3ffa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:49.876797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64671dc-8694-4887-9df7-4150d7f3ffa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:49.877080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64671dc-8694-4887-9df7-4150d7f3ffa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:49.877355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64671dc-8694-4887-9df7-4150d7f3ffa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:49.878105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64671dc-8694-4887-9df7-4150d7f3ffa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:49.880037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64671dc-8694-4887-9df7-4150d7f3ffa2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:49.890011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 947/1873] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:49 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:49.894395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f72f6-a826-49c7-86a1-0c7a9b7f4b74 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:49.912946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 927/1874] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:49 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:49.917023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64671dc-8694-4887-9df7-4150d7f3ffa2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:49.967959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f72f6-a826-49c7-86a1-0c7a9b7f4b74 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:49.991170 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:50.002542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64671dc-8694-4887-9df7-4150d7f3ffa2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.008124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68142d48-b27e-457a-86f9-9c8c6ee36376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.008826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68142d48-b27e-457a-86f9-9c8c6ee36376 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.009143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68142d48-b27e-457a-86f9-9c8c6ee36376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.009426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68142d48-b27e-457a-86f9-9c8c6ee36376 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.010411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68142d48-b27e-457a-86f9-9c8c6ee36376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.012451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68142d48-b27e-457a-86f9-9c8c6ee36376 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.020026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:50.035423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35ba50-9689-40ec-9eb6-c4b836ee649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.036160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35ba50-9689-40ec-9eb6-c4b836ee649b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.036448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35ba50-9689-40ec-9eb6-c4b836ee649b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.036730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35ba50-9689-40ec-9eb6-c4b836ee649b admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.037440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35ba50-9689-40ec-9eb6-c4b836ee649b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.039988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35ba50-9689-40ec-9eb6-c4b836ee649b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.040593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 928/1875] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:49 2022] POST /identity/v3/users => generated 282 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:14:50.044694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68142d48-b27e-457a-86f9-9c8c6ee36376 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.056418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68142d48-b27e-457a-86f9-9c8c6ee36376 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.063665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a56ea-688f-4cc8-a597-28434d3f57c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.064362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a56ea-688f-4cc8-a597-28434d3f57c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.064654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a56ea-688f-4cc8-a597-28434d3f57c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.064940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a56ea-688f-4cc8-a597-28434d3f57c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.065540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 948/1876] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:49 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:14:50.065842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a56ea-688f-4cc8-a597-28434d3f57c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.067485 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a56ea-688f-4cc8-a597-28434d3f57c2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.069514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35ba50-9689-40ec-9eb6-c4b836ee649b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.079414 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 929/1877] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:50.082270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35ba50-9689-40ec-9eb6-c4b836ee649b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.083286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a56ea-688f-4cc8-a597-28434d3f57c2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.087535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d52ab4-3dbf-46d5-918f-cdaafc2e4879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.088225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d52ab4-3dbf-46d5-918f-cdaafc2e4879 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.088653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d52ab4-3dbf-46d5-918f-cdaafc2e4879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.088941 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d52ab4-3dbf-46d5-918f-cdaafc2e4879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.089687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d52ab4-3dbf-46d5-918f-cdaafc2e4879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.091687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d52ab4-3dbf-46d5-918f-cdaafc2e4879 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.094524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a56ea-688f-4cc8-a597-28434d3f57c2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.100896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d02289-dbe4-454b-8e56-2f7adc906d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.101583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d02289-dbe4-454b-8e56-2f7adc906d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.101872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d02289-dbe4-454b-8e56-2f7adc906d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.102158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d02289-dbe4-454b-8e56-2f7adc906d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.102901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d02289-dbe4-454b-8e56-2f7adc906d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.106323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 949/1878] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:50.106501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d02289-dbe4-454b-8e56-2f7adc906d52 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.107662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d52ab4-3dbf-46d5-918f-cdaafc2e4879 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.116818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 930/1879] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:50.122606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d52ab4-3dbf-46d5-918f-cdaafc2e4879 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.123227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d02289-dbe4-454b-8e56-2f7adc906d52 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.127333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b2cca-34e9-4a2b-8e79-e1ea064477e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.128293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b2cca-34e9-4a2b-8e79-e1ea064477e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.128523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b2cca-34e9-4a2b-8e79-e1ea064477e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.128948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b2cca-34e9-4a2b-8e79-e1ea064477e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.129730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b2cca-34e9-4a2b-8e79-e1ea064477e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.131379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b2cca-34e9-4a2b-8e79-e1ea064477e7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.134437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d02289-dbe4-454b-8e56-2f7adc906d52 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.139652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cb056-75f7-4217-9a7d-62476757b8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.140606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cb056-75f7-4217-9a7d-62476757b8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.140893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cb056-75f7-4217-9a7d-62476757b8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.141279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cb056-75f7-4217-9a7d-62476757b8d8 admin admin] PATH_INFO: `/v3/projects/95a7d6e0d8a04096b6af95c1f2c3bbf3/users/85257f24169a421fa5a170f8a84a6a7d/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.144872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 950/1880] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:50.148998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b2cca-34e9-4a2b-8e79-e1ea064477e7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.160071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cb056-75f7-4217-9a7d-62476757b8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a7d6e0d8a04096b6af95c1f2c3bbf3, user_id=85257f24169a421fa5a170f8a84a6a7d, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.160758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b2cca-34e9-4a2b-8e79-e1ea064477e7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.163452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cb056-75f7-4217-9a7d-62476757b8d8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.165973 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef1b5e-d2f9-46dc-81e7-f4f341ce4e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.166882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef1b5e-d2f9-46dc-81e7-f4f341ce4e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.167182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef1b5e-d2f9-46dc-81e7-f4f341ce4e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.167500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef1b5e-d2f9-46dc-81e7-f4f341ce4e5c admin admin] PATH_INFO: `/v3/projects/f96f5bb355964b61ac770d29d3bdf456/users/4d56e4c4e6e84022beb72e043c07c751/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.187505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 931/1881] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:50 2022] PUT /identity/v3/projects/95a7d6e0d8a04096b6af95c1f2c3bbf3/users/85257f24169a421fa5a170f8a84a6a7d/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:50.196268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef1b5e-d2f9-46dc-81e7-f4f341ce4e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96f5bb355964b61ac770d29d3bdf456, user_id=4d56e4c4e6e84022beb72e043c07c751, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.198851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39d066-c3bb-4140-b19d-04d49aab88ae None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.198971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef1b5e-d2f9-46dc-81e7-f4f341ce4e5c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.199152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39d066-c3bb-4140-b19d-04d49aab88ae None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.199475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39d066-c3bb-4140-b19d-04d49aab88ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.223344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 951/1882] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:50 2022] PUT /identity/v3/projects/f96f5bb355964b61ac770d29d3bdf456/users/4d56e4c4e6e84022beb72e043c07c751/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:50.225539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5e0a6-5a08-4ac7-bd19-65eb63dfd8e3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.225845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5e0a6-5a08-4ac7-bd19-65eb63dfd8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.226367 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5e0a6-5a08-4ac7-bd19-65eb63dfd8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.229739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8e39d066-c3bb-4140-b19d-04d49aab88ae None None] MFA Rules not processed for user `85257f24169a421fa5a170f8a84a6a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:50.231337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8e39d066-c3bb-4140-b19d-04d49aab88ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:50.256023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f8a5e0a6-5a08-4ac7-bd19-65eb63dfd8e3 None None] MFA Rules not processed for user `4d56e4c4e6e84022beb72e043c07c751`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:50.257291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f8a5e0a6-5a08-4ac7-bd19-65eb63dfd8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:50.308911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 932/1883] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:50.314194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39d066-c3bb-4140-b19d-04d49aab88ae None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.343838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 952/1884] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:50.349381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5e0a6-5a08-4ac7-bd19-65eb63dfd8e3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.383576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39d066-c3bb-4140-b19d-04d49aab88ae None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.388625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4b323-23f4-4409-8d5b-27a26864810f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.389531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4b323-23f4-4409-8d5b-27a26864810f admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.389829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4b323-23f4-4409-8d5b-27a26864810f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.390117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4b323-23f4-4409-8d5b-27a26864810f admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.390976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4b323-23f4-4409-8d5b-27a26864810f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.393501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4b323-23f4-4409-8d5b-27a26864810f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.413391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:50.423314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 933/1885] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:50 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:14:50.423507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5e0a6-5a08-4ac7-bd19-65eb63dfd8e3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.426022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4b323-23f4-4409-8d5b-27a26864810f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.446565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:50.483619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302f780-1f88-4513-b36f-f45602483d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.484302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302f780-1f88-4513-b36f-f45602483d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.484592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302f780-1f88-4513-b36f-f45602483d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.484879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302f780-1f88-4513-b36f-f45602483d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.485614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302f780-1f88-4513-b36f-f45602483d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.487558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302f780-1f88-4513-b36f-f45602483d16 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.492003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4b323-23f4-4409-8d5b-27a26864810f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.496593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cceb22-6e67-4410-b4d7-cf515da12996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.497280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cceb22-6e67-4410-b4d7-cf515da12996 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.497566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cceb22-6e67-4410-b4d7-cf515da12996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.497844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cceb22-6e67-4410-b4d7-cf515da12996 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.498542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cceb22-6e67-4410-b4d7-cf515da12996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.500533 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cceb22-6e67-4410-b4d7-cf515da12996 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.520335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 953/1886] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:50 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:14:50.521422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302f780-1f88-4513-b36f-f45602483d16 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.522197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 934/1887] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:50 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:14:50.526161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cceb22-6e67-4410-b4d7-cf515da12996 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.537031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cceb22-6e67-4410-b4d7-cf515da12996 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.585569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302f780-1f88-4513-b36f-f45602483d16 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.590454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329937e3-1115-49d9-b170-0ec37e771a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.591145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329937e3-1115-49d9-b170-0ec37e771a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.591471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329937e3-1115-49d9-b170-0ec37e771a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.591759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329937e3-1115-49d9-b170-0ec37e771a2f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.592466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329937e3-1115-49d9-b170-0ec37e771a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.594416 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329937e3-1115-49d9-b170-0ec37e771a2f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.605205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20cef6-9949-4683-ab21-d298b54a96a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.606065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20cef6-9949-4683-ab21-d298b54a96a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.606298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20cef6-9949-4683-ab21-d298b54a96a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.606658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20cef6-9949-4683-ab21-d298b54a96a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.607626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20cef6-9949-4683-ab21-d298b54a96a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.609432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20cef6-9949-4683-ab21-d298b54a96a0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.620919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 954/1888] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:50 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:14:50.622881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 935/1889] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:50.623001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329937e3-1115-49d9-b170-0ec37e771a2f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.626434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20cef6-9949-4683-ab21-d298b54a96a0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.634315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329937e3-1115-49d9-b170-0ec37e771a2f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.637277 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20cef6-9949-4683-ab21-d298b54a96a0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.639129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07857bc-9ece-4a07-9c2a-8c9ee59ff21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.639846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07857bc-9ece-4a07-9c2a-8c9ee59ff21e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.640139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07857bc-9ece-4a07-9c2a-8c9ee59ff21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.640452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07857bc-9ece-4a07-9c2a-8c9ee59ff21e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.641198 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07857bc-9ece-4a07-9c2a-8c9ee59ff21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.642697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f824d2-9cf9-4d46-9857-29af43224c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.643455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07857bc-9ece-4a07-9c2a-8c9ee59ff21e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.643627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f824d2-9cf9-4d46-9857-29af43224c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.643912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f824d2-9cf9-4d46-9857-29af43224c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.644178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f824d2-9cf9-4d46-9857-29af43224c98 admin admin] PATH_INFO: `/v3/projects/f7dc2b8628704b68a6212cdbfe60cabc/users/2f810903f0524f9da8fc7e2f66f5bb6a/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.654313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 955/1890] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:50.658596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07857bc-9ece-4a07-9c2a-8c9ee59ff21e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.660234 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f824d2-9cf9-4d46-9857-29af43224c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7dc2b8628704b68a6212cdbfe60cabc, user_id=2f810903f0524f9da8fc7e2f66f5bb6a, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.662974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f824d2-9cf9-4d46-9857-29af43224c98 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.669055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07857bc-9ece-4a07-9c2a-8c9ee59ff21e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.675226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01276c89-6a96-4834-bdf1-f290b08f27d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.676163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276c89-6a96-4834-bdf1-f290b08f27d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.676451 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276c89-6a96-4834-bdf1-f290b08f27d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.676734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276c89-6a96-4834-bdf1-f290b08f27d6 admin admin] PATH_INFO: `/v3/projects/e1d99921e9764c469c8a4aca9c593c92/users/b264fcf4510648be8dc70db1b19a203e/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.676991 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 936/1891] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:50 2022] PUT /identity/v3/projects/f7dc2b8628704b68a6212cdbfe60cabc/users/2f810903f0524f9da8fc7e2f66f5bb6a/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:50.681104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f824d2-9cf9-4d46-9857-29af43224c98 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.695967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01276c89-6a96-4834-bdf1-f290b08f27d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d99921e9764c469c8a4aca9c593c92, user_id=b264fcf4510648be8dc70db1b19a203e, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.698685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01276c89-6a96-4834-bdf1-f290b08f27d6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.713749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 956/1892] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:50 2022] PUT /identity/v3/projects/e1d99921e9764c469c8a4aca9c593c92/users/b264fcf4510648be8dc70db1b19a203e/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:50.717670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01276c89-6a96-4834-bdf1-f290b08f27d6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.767737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f824d2-9cf9-4d46-9857-29af43224c98 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.796839 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:50.815593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:50.822309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01276c89-6a96-4834-bdf1-f290b08f27d6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.827932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6987013-8512-4c96-b2ac-4b8ddc50a6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.829837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6987013-8512-4c96-b2ac-4b8ddc50a6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.830138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6987013-8512-4c96-b2ac-4b8ddc50a6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.830413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6987013-8512-4c96-b2ac-4b8ddc50a6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.831166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6987013-8512-4c96-b2ac-4b8ddc50a6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.833050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6987013-8512-4c96-b2ac-4b8ddc50a6ab admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.854220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 957/1893] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:50.855545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6987013-8512-4c96-b2ac-4b8ddc50a6ab admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.866776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6987013-8512-4c96-b2ac-4b8ddc50a6ab admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.867763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86723185-f2ea-4233-b4fd-48be9566af56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.868552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86723185-f2ea-4233-b4fd-48be9566af56 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.868869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86723185-f2ea-4233-b4fd-48be9566af56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.869143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86723185-f2ea-4233-b4fd-48be9566af56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.869893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86723185-f2ea-4233-b4fd-48be9566af56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.871753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86723185-f2ea-4233-b4fd-48be9566af56 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.872342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afac75f-a2e3-476c-b9dc-effb9f36243c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.873301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afac75f-a2e3-476c-b9dc-effb9f36243c admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.873597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afac75f-a2e3-476c-b9dc-effb9f36243c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.873879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afac75f-a2e3-476c-b9dc-effb9f36243c admin admin] PATH_INFO: `/v3/projects/e1d99921e9764c469c8a4aca9c593c92/users/b264fcf4510648be8dc70db1b19a203e/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.876495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afac75f-a2e3-476c-b9dc-effb9f36243c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d99921e9764c469c8a4aca9c593c92, user_id=b264fcf4510648be8dc70db1b19a203e, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.879192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afac75f-a2e3-476c-b9dc-effb9f36243c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.886982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 937/1894] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:50.888488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86723185-f2ea-4233-b4fd-48be9566af56 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:50.895667 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 958/1895] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:50 2022] PUT /identity/v3/projects/e1d99921e9764c469c8a4aca9c593c92/users/b264fcf4510648be8dc70db1b19a203e/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:50.902026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86723185-f2ea-4233-b4fd-48be9566af56 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:50.906682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ac5b8-8330-45eb-a343-daa51860a66a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.906984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ac5b8-8330-45eb-a343-daa51860a66a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.907349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ac5b8-8330-45eb-a343-daa51860a66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.937059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-978ac5b8-8330-45eb-a343-daa51860a66a None None] MFA Rules not processed for user `b264fcf4510648be8dc70db1b19a203e`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:50.938312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-978ac5b8-8330-45eb-a343-daa51860a66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:50.970193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6ec27-5d95-4677-b6e5-ecb87985e6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:50.971108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6ec27-5d95-4677-b6e5-ecb87985e6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:50.971461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6ec27-5d95-4677-b6e5-ecb87985e6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:50.973431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6ec27-5d95-4677-b6e5-ecb87985e6cf admin admin] PATH_INFO: `/v3/projects/f7dc2b8628704b68a6212cdbfe60cabc/users/2f810903f0524f9da8fc7e2f66f5bb6a/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:50.976445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6ec27-5d95-4677-b6e5-ecb87985e6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7dc2b8628704b68a6212cdbfe60cabc, user_id=2f810903f0524f9da8fc7e2f66f5bb6a, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:50.979160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6ec27-5d95-4677-b6e5-ecb87985e6cf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:50.994762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 938/1896] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:50 2022] PUT /identity/v3/projects/f7dc2b8628704b68a6212cdbfe60cabc/users/2f810903f0524f9da8fc7e2f66f5bb6a/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:51.005741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476da934-1e1d-4303-9071-16d3b7e6f4bf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:51.006040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476da934-1e1d-4303-9071-16d3b7e6f4bf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:51.006329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476da934-1e1d-4303-9071-16d3b7e6f4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:51.023213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 959/1897] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:51.035122 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ac5b8-8330-45eb-a343-daa51860a66a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:51.036838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-978ac5b8-8330-45eb-a343-daa51860a66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:51.040053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-476da934-1e1d-4303-9071-16d3b7e6f4bf None None] MFA Rules not processed for user `2f810903f0524f9da8fc7e2f66f5bb6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:51.041206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-476da934-1e1d-4303-9071-16d3b7e6f4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:51.101746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ac5b8-8330-45eb-a343-daa51860a66a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:51.130286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 939/1898] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:51 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:14:51.144088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476da934-1e1d-4303-9071-16d3b7e6f4bf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:51.145500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-476da934-1e1d-4303-9071-16d3b7e6f4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:51.158283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5ef3e-e9f1-4d18-a1ae-422eb2af6193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:51.158959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5ef3e-e9f1-4d18-a1ae-422eb2af6193 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:51.159249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5ef3e-e9f1-4d18-a1ae-422eb2af6193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:51.159561 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5ef3e-e9f1-4d18-a1ae-422eb2af6193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:51.183420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:51.198817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:51.212247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476da934-1e1d-4303-9071-16d3b7e6f4bf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:51.217477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ab1d9-9dda-4398-a391-24db26fc6706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:51.218052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ab1d9-9dda-4398-a391-24db26fc6706 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:51.218348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ab1d9-9dda-4398-a391-24db26fc6706 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:51.218631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ab1d9-9dda-4398-a391-24db26fc6706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:51.237530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ab1d9-9dda-4398-a391-24db26fc6706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:51.239001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5ef3e-e9f1-4d18-a1ae-422eb2af6193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:51.239994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ab1d9-9dda-4398-a391-24db26fc6706 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:51.240800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5ef3e-e9f1-4d18-a1ae-422eb2af6193 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:51.250433 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 940/1899] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:51 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:51.251723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 960/1900] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:51.332590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ab1d9-9dda-4398-a391-24db26fc6706 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:51.343125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ab1d9-9dda-4398-a391-24db26fc6706 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:51.345819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5ef3e-e9f1-4d18-a1ae-422eb2af6193 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:51.350916 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15592c7a-dfa4-4948-9bb1-b46e18db223f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:51.353453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15592c7a-dfa4-4948-9bb1-b46e18db223f service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:51.354585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15592c7a-dfa4-4948-9bb1-b46e18db223f service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:51.355247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15592c7a-dfa4-4948-9bb1-b46e18db223f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:51.355971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5ef3e-e9f1-4d18-a1ae-422eb2af6193 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:51.360905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb89bd-0fe4-481c-8efb-0a7182bc533b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:51.361576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb89bd-0fe4-481c-8efb-0a7182bc533b service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:51.362223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb89bd-0fe4-481c-8efb-0a7182bc533b service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:51.362518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb89bd-0fe4-481c-8efb-0a7182bc533b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:51.429680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15592c7a-dfa4-4948-9bb1-b46e18db223f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:51.431144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15592c7a-dfa4-4948-9bb1-b46e18db223f service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:51.432691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb89bd-0fe4-481c-8efb-0a7182bc533b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:51.434534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb89bd-0fe4-481c-8efb-0a7182bc533b service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:51.441792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 941/1901] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:51.445011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 961/1902] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:51.764425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15592c7a-dfa4-4948-9bb1-b46e18db223f service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:51.838640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15592c7a-dfa4-4948-9bb1-b46e18db223f service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:51.849239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:51.850747 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:51.851245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:51.851784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] PATH_INFO: `/v3/users/4d56e4c4e6e84022beb72e043c07c751` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:51.853920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d56e4c4e6e84022beb72e043c07c751)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:51.857435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:51.920855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d56e4c4e6e84022beb72e043c07c751'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:51.932415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d56e4c4e6e84022beb72e043c07c751'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:51.945464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d56e4c4e6e84022beb72e043c07c751'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:51.962830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d56e4c4e6e84022beb72e043c07c751'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:51.964056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d56e4c4e6e84022beb72e043c07c751'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:51.973301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 942/1903] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:51 2022] DELETE /identity/v3/users/4d56e4c4e6e84022beb72e043c07c751 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:51.979471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8b54-64f2-4e52-a6bc-156c576bc027 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:51.979992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8b54-64f2-4e52-a6bc-156c576bc027 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:51.980554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8b54-64f2-4e52-a6bc-156c576bc027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:52.011205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e82e8b54-64f2-4e52-a6bc-156c576bc027 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:52.013534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e82e8b54-64f2-4e52-a6bc-156c576bc027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:52.166786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 962/1904] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:52.181345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:52.183078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:52.193409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:52.294143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f366a-40ba-4ed9-8ac3-f3d2a0d4deaf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:52.299248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b5f2e-141b-4604-bdc0-9d427966d57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:52.300871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b5f2e-141b-4604-bdc0-9d427966d57b service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:52.301368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b5f2e-141b-4604-bdc0-9d427966d57b service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:52.301828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b5f2e-141b-4604-bdc0-9d427966d57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:52.314750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b5f2e-141b-4604-bdc0-9d427966d57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:52.316629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b5f2e-141b-4604-bdc0-9d427966d57b service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:52.334650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 943/1905] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:52.932251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e8b54-64f2-4e52-a6bc-156c576bc027 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:52.933783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e82e8b54-64f2-4e52-a6bc-156c576bc027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:52.949309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e8b54-64f2-4e52-a6bc-156c576bc027 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:52.955521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:52.956588 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:52.956882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:52.957218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] PATH_INFO: `/v3/users/a1710cca0d84444d9b90a90eaf7c55ef` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:52.960085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1710cca0d84444d9b90a90eaf7c55ef)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:52.963854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:53.113902 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1710cca0d84444d9b90a90eaf7c55ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.117142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1710cca0d84444d9b90a90eaf7c55ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.123942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:53.134553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1710cca0d84444d9b90a90eaf7c55ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.158712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1710cca0d84444d9b90a90eaf7c55ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.185461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1710cca0d84444d9b90a90eaf7c55ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.208345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 963/1906] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:52 2022] DELETE /identity/v3/users/a1710cca0d84444d9b90a90eaf7c55ef => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:53.213573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e93a3-4efd-4f8b-b27c-82c2ccb6b72b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:53.214058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e93a3-4efd-4f8b-b27c-82c2ccb6b72b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:53.214058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e93a3-4efd-4f8b-b27c-82c2ccb6b72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:53.238690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:53.240797 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:53.253770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-785e93a3-4efd-4f8b-b27c-82c2ccb6b72b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:53.255031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-785e93a3-4efd-4f8b-b27c-82c2ccb6b72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:53.343090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859ed01-498c-433f-b314-8300048bd267 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:53.362943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:53.364675 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:53.366330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:53.366819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] PATH_INFO: `/v3/projects/f96f5bb355964b61ac770d29d3bdf456` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:53.370145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96f5bb355964b61ac770d29d3bdf456)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:53.375074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:53.398659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 944/1907] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:53.452323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e93a3-4efd-4f8b-b27c-82c2ccb6b72b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:53.454001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-785e93a3-4efd-4f8b-b27c-82c2ccb6b72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:53.492032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96f5bb355964b61ac770d29d3bdf456', 'request_id': 'req-e5b0c1f2-9a37-4cc1-a526-5f0213089150'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.499796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96f5bb355964b61ac770d29d3bdf456', 'request_id': 'req-e5b0c1f2-9a37-4cc1-a526-5f0213089150'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.517618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96f5bb355964b61ac770d29d3bdf456', 'request_id': 'req-e5b0c1f2-9a37-4cc1-a526-5f0213089150'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.540001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] The token cache is being invalidate because project f96f5bb355964b61ac770d29d3bdf456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:53.540503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.546745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.551710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 964/1908] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:53 2022] DELETE /identity/v3/projects/f96f5bb355964b61ac770d29d3bdf456 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:53.552901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:53.555720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:53.620522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e93a3-4efd-4f8b-b27c-82c2ccb6b72b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:53.748530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:53.755992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0c1f2-9a37-4cc1-a526-5f0213089150 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:53.766580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:53.768534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:53.769332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:53.769944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f393ab-eb17-4dcf-a7aa-8f695c6240b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:53.770118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] PATH_INFO: `/v3/users/b264fcf4510648be8dc70db1b19a203e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:53.771362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f393ab-eb17-4dcf-a7aa-8f695c6240b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:53.771802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f393ab-eb17-4dcf-a7aa-8f695c6240b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:53.772235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f393ab-eb17-4dcf-a7aa-8f695c6240b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:53.775771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b264fcf4510648be8dc70db1b19a203e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:53.778079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:53.778343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-46f393ab-eb17-4dcf-a7aa-8f695c6240b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:53.806995 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f393ab-eb17-4dcf-a7aa-8f695c6240b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:53.814033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f393ab-eb17-4dcf-a7aa-8f695c6240b1 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:53.843722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 945/1909] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:53.872069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b264fcf4510648be8dc70db1b19a203e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.873316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b264fcf4510648be8dc70db1b19a203e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.888585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b264fcf4510648be8dc70db1b19a203e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.920670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b264fcf4510648be8dc70db1b19a203e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.934895 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b264fcf4510648be8dc70db1b19a203e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:53.944827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 965/1910] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:53 2022] DELETE /identity/v3/users/b264fcf4510648be8dc70db1b19a203e => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:54.757424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f393ab-eb17-4dcf-a7aa-8f695c6240b1 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:54.788777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:54.797021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:54.800092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:54.810603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f393ab-eb17-4dcf-a7aa-8f695c6240b1 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:54.993428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707c059-010e-4d9f-97a0-36136e8120f4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:55.005199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:55.005875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:55.006231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:55.006744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:55.007216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] PATH_INFO: `/v3/projects/dba5fa0382bc4a69ab7732ea8171c09b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:55.008003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:55.008679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba5fa0382bc4a69ab7732ea8171c09b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:55.009013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:55.009652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] PATH_INFO: `/v3/projects/e1d99921e9764c469c8a4aca9c593c92` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:55.010836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:55.012140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d99921e9764c469c8a4aca9c593c92)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:55.017568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:55.102990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba5fa0382bc4a69ab7732ea8171c09b', 'request_id': 'req-8d04e607-fa8f-4291-afde-a38d5957950e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.105583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d99921e9764c469c8a4aca9c593c92', 'request_id': 'req-f5d4f90e-5186-4b94-a9ad-846112687eaf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.115968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba5fa0382bc4a69ab7732ea8171c09b', 'request_id': 'req-8d04e607-fa8f-4291-afde-a38d5957950e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.131244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d99921e9764c469c8a4aca9c593c92', 'request_id': 'req-f5d4f90e-5186-4b94-a9ad-846112687eaf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.132624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba5fa0382bc4a69ab7732ea8171c09b', 'request_id': 'req-8d04e607-fa8f-4291-afde-a38d5957950e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.147600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d99921e9764c469c8a4aca9c593c92', 'request_id': 'req-f5d4f90e-5186-4b94-a9ad-846112687eaf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.156567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] The token cache is being invalidate because project dba5fa0382bc4a69ab7732ea8171c09b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:55.156795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.161230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.164969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 966/1911] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:54 2022] DELETE /identity/v3/projects/dba5fa0382bc4a69ab7732ea8171c09b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:55.174790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:55.178749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:55.179099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] The token cache is being invalidate because project e1d99921e9764c469c8a4aca9c593c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:55.179434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.181016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.186489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 946/1912] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:54 2022] DELETE /identity/v3/projects/e1d99921e9764c469c8a4aca9c593c92 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:55.188104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:55.193819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:55.211941 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:55.373615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04e607-fa8f-4291-afde-a38d5957950e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:55.443389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4f90e-5186-4b94-a9ad-846112687eaf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:55.466137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:55.467620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:55.468342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:55.468811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] PATH_INFO: `/v3/users/0d64da65cf9b45d5a3233414538d0b43` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:55.475883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d64da65cf9b45d5a3233414538d0b43)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:55.488623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:55.574574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:55.611955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d64da65cf9b45d5a3233414538d0b43'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.625282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:55.626464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:55.626872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:55.626872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] PATH_INFO: `/v3/users/85257f24169a421fa5a170f8a84a6a7d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:55.629094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85257f24169a421fa5a170f8a84a6a7d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:55.632250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d64da65cf9b45d5a3233414538d0b43'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.632406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:55.654014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d64da65cf9b45d5a3233414538d0b43'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.674781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d64da65cf9b45d5a3233414538d0b43'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.675925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1104ede0-0f5b-4616-b67c-c7d133c27669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d64da65cf9b45d5a3233414538d0b43'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.690606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 947/1913] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:55 2022] DELETE /identity/v3/users/0d64da65cf9b45d5a3233414538d0b43 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:55.692473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e102e-4116-4ced-a97f-46a6430b4b15 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:55.693575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e102e-4116-4ced-a97f-46a6430b4b15 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:55.693873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e102e-4116-4ced-a97f-46a6430b4b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:55.745307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-601e102e-4116-4ced-a97f-46a6430b4b15 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:55.753698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-601e102e-4116-4ced-a97f-46a6430b4b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:55.770623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85257f24169a421fa5a170f8a84a6a7d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.780163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85257f24169a421fa5a170f8a84a6a7d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.837939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85257f24169a421fa5a170f8a84a6a7d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.881699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85257f24169a421fa5a170f8a84a6a7d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.888109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 948/1914] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:55.898004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a609ddc-a6c0-4c4d-a575-26b58e823334 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:55.899071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a609ddc-a6c0-4c4d-a575-26b58e823334 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:55.899488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a609ddc-a6c0-4c4d-a575-26b58e823334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:55.909070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c6ace79-d0d7-4936-ba9c-80698e30d026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85257f24169a421fa5a170f8a84a6a7d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:55.924336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0a609ddc-a6c0-4c4d-a575-26b58e823334 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:55.925853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0a609ddc-a6c0-4c4d-a575-26b58e823334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:55.932940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 967/1915] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:55 2022] DELETE /identity/v3/users/85257f24169a421fa5a170f8a84a6a7d => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:55.940071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dae8a-c54d-45f4-9db5-83b77afbb064 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:55.940890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dae8a-c54d-45f4-9db5-83b77afbb064 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:55.941567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dae8a-c54d-45f4-9db5-83b77afbb064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:56.010257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-939dae8a-c54d-45f4-9db5-83b77afbb064 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:56.011397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 949/1916] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:56.011966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a609ddc-a6c0-4c4d-a575-26b58e823334 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:56.017549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-939dae8a-c54d-45f4-9db5-83b77afbb064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:56.023601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a609ddc-a6c0-4c4d-a575-26b58e823334 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:56.031206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9933d-ba71-4327-8ca3-7cb6b63ed054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:56.031828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9933d-ba71-4327-8ca3-7cb6b63ed054 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:56.032463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9933d-ba71-4327-8ca3-7cb6b63ed054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:56.032959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9933d-ba71-4327-8ca3-7cb6b63ed054 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:56.034354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9933d-ba71-4327-8ca3-7cb6b63ed054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:56.034828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9933d-ba71-4327-8ca3-7cb6b63ed054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:56.036963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9933d-ba71-4327-8ca3-7cb6b63ed054 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:56.055958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 950/1917] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:56.061427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9933d-ba71-4327-8ca3-7cb6b63ed054 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:56.077107 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 968/1918] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:56.093216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dae8a-c54d-45f4-9db5-83b77afbb064 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:56.097035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-939dae8a-c54d-45f4-9db5-83b77afbb064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:56.106979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9933d-ba71-4327-8ca3-7cb6b63ed054 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:56.155597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfabe27-0de9-443e-b95e-3e74dd4fd555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:56.168693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfabe27-0de9-443e-b95e-3e74dd4fd555 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:56.169234 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfabe27-0de9-443e-b95e-3e74dd4fd555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:56.173375 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfabe27-0de9-443e-b95e-3e74dd4fd555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:56.176188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfabe27-0de9-443e-b95e-3e74dd4fd555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:56.184723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:56.202297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfabe27-0de9-443e-b95e-3e74dd4fd555 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:56.279104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dae8a-c54d-45f4-9db5-83b77afbb064 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:56.288615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684df4c-4f1b-4cac-b3d6-be32110e7cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:56.289940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684df4c-4f1b-4cac-b3d6-be32110e7cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:56.290484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684df4c-4f1b-4cac-b3d6-be32110e7cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:56.291202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684df4c-4f1b-4cac-b3d6-be32110e7cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:56.312266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684df4c-4f1b-4cac-b3d6-be32110e7cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:56.315381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684df4c-4f1b-4cac-b3d6-be32110e7cd2 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:56.331567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 951/1919] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:56 2022] POST /identity/v3/projects => generated 377 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:56.342837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 969/1920] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:56.349458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfabe27-0de9-443e-b95e-3e74dd4fd555 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:56.405162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:56.501400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfabe27-0de9-443e-b95e-3e74dd4fd555 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:56.509645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003fcf4-6f70-4fab-82fc-81931dc7e386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:56.510608 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003fcf4-6f70-4fab-82fc-81931dc7e386 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:56.511060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003fcf4-6f70-4fab-82fc-81931dc7e386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:56.511481 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003fcf4-6f70-4fab-82fc-81931dc7e386 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:56.512299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003fcf4-6f70-4fab-82fc-81931dc7e386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:56.514512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003fcf4-6f70-4fab-82fc-81931dc7e386 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:56.557035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 952/1921] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:56 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:56.562956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684df4c-4f1b-4cac-b3d6-be32110e7cd2 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:56.575197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684df4c-4f1b-4cac-b3d6-be32110e7cd2 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:56.580439 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3965d62-3783-40a5-8d2c-2b607674083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:56.581859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3965d62-3783-40a5-8d2c-2b607674083d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:56.582054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3965d62-3783-40a5-8d2c-2b607674083d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:56.582335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3965d62-3783-40a5-8d2c-2b607674083d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:56.583071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3965d62-3783-40a5-8d2c-2b607674083d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:56.584826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3965d62-3783-40a5-8d2c-2b607674083d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:56.598933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 970/1922] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:56.604416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003fcf4-6f70-4fab-82fc-81931dc7e386 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:56.622274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003fcf4-6f70-4fab-82fc-81931dc7e386 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:56.636919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ea5ad-4591-42e2-b945-2aac08bd766c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:56.638310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ea5ad-4591-42e2-b945-2aac08bd766c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:56.639043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ea5ad-4591-42e2-b945-2aac08bd766c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:56.640107 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ea5ad-4591-42e2-b945-2aac08bd766c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:56.641795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ea5ad-4591-42e2-b945-2aac08bd766c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:56.647386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ea5ad-4591-42e2-b945-2aac08bd766c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:56.683157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 953/1923] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:56.684085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3965d62-3783-40a5-8d2c-2b607674083d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:56.704555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3965d62-3783-40a5-8d2c-2b607674083d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:56.728480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2c91d-fe45-40da-8eb6-c1626509b311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:56.729559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2c91d-fe45-40da-8eb6-c1626509b311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:56.731007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2c91d-fe45-40da-8eb6-c1626509b311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:56.731007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2c91d-fe45-40da-8eb6-c1626509b311 admin admin] PATH_INFO: `/v3/projects/bc1a73b47e7a41789b045738110dee21/users/b375915231a14fc299367015ecfbf967/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:56.771287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2c91d-fe45-40da-8eb6-c1626509b311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1a73b47e7a41789b045738110dee21, user_id=b375915231a14fc299367015ecfbf967, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:56.777861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2c91d-fe45-40da-8eb6-c1626509b311 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:56.814293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 971/1924] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:56 2022] PUT /identity/v3/projects/bc1a73b47e7a41789b045738110dee21/users/b375915231a14fc299367015ecfbf967/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:56.830039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ffe8d-53c2-4930-a226-7be7f70e8e4c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:56.830645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ffe8d-53c2-4930-a226-7be7f70e8e4c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:56.831220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ffe8d-53c2-4930-a226-7be7f70e8e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:56.904623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-293ffe8d-53c2-4930-a226-7be7f70e8e4c None None] MFA Rules not processed for user `b375915231a14fc299367015ecfbf967`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:56.906797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-293ffe8d-53c2-4930-a226-7be7f70e8e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:57.001937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2c91d-fe45-40da-8eb6-c1626509b311 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:57.004678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0ac2c91d-fe45-40da-8eb6-c1626509b311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:57.037547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 954/1925] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:57.074296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ffe8d-53c2-4930-a226-7be7f70e8e4c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:57.087086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-293ffe8d-53c2-4930-a226-7be7f70e8e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:57.155913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2c91d-fe45-40da-8eb6-c1626509b311 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:57.163951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:57.165041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:57.165237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:57.165520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] PATH_INFO: `/v3/projects/ac17974f6fd848389abe1553e561be8d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:57.166722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac17974f6fd848389abe1553e561be8d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:57.169564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:57.200318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ffe8d-53c2-4930-a226-7be7f70e8e4c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:57.209791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:57.211222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:57.211739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:57.212209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] PATH_INFO: `/v3/projects/95a7d6e0d8a04096b6af95c1f2c3bbf3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:57.213985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a7d6e0d8a04096b6af95c1f2c3bbf3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:57.217824 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:57.245972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac17974f6fd848389abe1553e561be8d', 'request_id': 'req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.256610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac17974f6fd848389abe1553e561be8d', 'request_id': 'req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.264746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac17974f6fd848389abe1553e561be8d', 'request_id': 'req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.284016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] The token cache is being invalidate because project ac17974f6fd848389abe1553e561be8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:57.284373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.285323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.287051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 972/1926] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:57 2022] DELETE /identity/v3/projects/ac17974f6fd848389abe1553e561be8d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:14:57.288514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:57.290655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:57.298193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:57.317827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a7d6e0d8a04096b6af95c1f2c3bbf3', 'request_id': 'req-59e34a9e-a82d-438a-9dd9-344c52071d1d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.327575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a7d6e0d8a04096b6af95c1f2c3bbf3', 'request_id': 'req-59e34a9e-a82d-438a-9dd9-344c52071d1d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.335324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a7d6e0d8a04096b6af95c1f2c3bbf3', 'request_id': 'req-59e34a9e-a82d-438a-9dd9-344c52071d1d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.355052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:57.368756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] The token cache is being invalidate because project 95a7d6e0d8a04096b6af95c1f2c3bbf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:57.369066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.369813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59e34a9e-a82d-438a-9dd9-344c52071d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.371560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 955/1927] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:57 2022] DELETE /identity/v3/projects/95a7d6e0d8a04096b6af95c1f2c3bbf3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:57.376819 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4b9a6-c74d-4a6c-9f2a-0deec5cacecf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:57.377520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4b9a6-c74d-4a6c-9f2a-0deec5cacecf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:57.377816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4b9a6-c74d-4a6c-9f2a-0deec5cacecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:57.399470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a8d4b9a6-c74d-4a6c-9f2a-0deec5cacecf None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:57.400723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a8d4b9a6-c74d-4a6c-9f2a-0deec5cacecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:57.441163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f10de-bb7e-4308-b5ac-04b8a63293b2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:57.536119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 956/1928] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:57.536119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4b9a6-c74d-4a6c-9f2a-0deec5cacecf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:57.536119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a8d4b9a6-c74d-4a6c-9f2a-0deec5cacecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:57.559133 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4b9a6-c74d-4a6c-9f2a-0deec5cacecf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:57.579923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:57.581346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:57.587694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:57.587694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] PATH_INFO: `/v3/users/2f810903f0524f9da8fc7e2f66f5bb6a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:57.608512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f810903f0524f9da8fc7e2f66f5bb6a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:57.615906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:57.632444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92fc25f9ad7c49c48a909d2d92ea0a66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:57.635307 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:57.643303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:57.647599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:57.659049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:57.749442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f810903f0524f9da8fc7e2f66f5bb6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.766522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f810903f0524f9da8fc7e2f66f5bb6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.788514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f810903f0524f9da8fc7e2f66f5bb6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.831850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f810903f0524f9da8fc7e2f66f5bb6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.833529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5377f84-dbbe-4ff3-a501-a7e04a919862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f810903f0524f9da8fc7e2f66f5bb6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:57.850102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:57.853704 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:57.864836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 957/1929] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:14:57 2022] DELETE /identity/v3/users/2f810903f0524f9da8fc7e2f66f5bb6a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:57.867486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c577fa-4e70-442d-8364-ea45cab7b910 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:57.868218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c577fa-4e70-442d-8364-ea45cab7b910 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:57.877855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c577fa-4e70-442d-8364-ea45cab7b910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:57.878527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 973/1930] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:14:57.942726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-03c577fa-4e70-442d-8364-ea45cab7b910 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:57.949472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03c577fa-4e70-442d-8364-ea45cab7b910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:58.004930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:58.053356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 958/1931] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:58.058835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:58.061581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:58.085987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c661c9-b5f7-46ad-a97f-d6781f837c2c service glance] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:58.111818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db760816-06b5-4623-a09a-791d60a72917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:58.112980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db760816-06b5-4623-a09a-791d60a72917 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:58.113401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db760816-06b5-4623-a09a-791d60a72917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:58.113786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db760816-06b5-4623-a09a-791d60a72917 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:58.115132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db760816-06b5-4623-a09a-791d60a72917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:58.115610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db760816-06b5-4623-a09a-791d60a72917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:58.117522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db760816-06b5-4623-a09a-791d60a72917 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:58.150518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c577fa-4e70-442d-8364-ea45cab7b910 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:58.160283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 974/1932] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:58.174613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c577fa-4e70-442d-8364-ea45cab7b910 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:58.191866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221f9cb-d3a9-4336-bb74-017651299a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:58.192556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221f9cb-d3a9-4336-bb74-017651299a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:58.193552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221f9cb-d3a9-4336-bb74-017651299a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:58.194523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221f9cb-d3a9-4336-bb74-017651299a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:58.196839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221f9cb-d3a9-4336-bb74-017651299a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:58.202356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221f9cb-d3a9-4336-bb74-017651299a8c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:58.274505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db760816-06b5-4623-a09a-791d60a72917 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:58.281016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 959/1933] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:58 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:58.287379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db760816-06b5-4623-a09a-791d60a72917 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:58.420604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48142d29-9c48-41f4-a840-3db3a18ea16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:58.421524 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48142d29-9c48-41f4-a840-3db3a18ea16a admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:58.421990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48142d29-9c48-41f4-a840-3db3a18ea16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:58.423667 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48142d29-9c48-41f4-a840-3db3a18ea16a admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:58.425815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48142d29-9c48-41f4-a840-3db3a18ea16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:58.430601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48142d29-9c48-41f4-a840-3db3a18ea16a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:58.444773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:58.483754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221f9cb-d3a9-4336-bb74-017651299a8c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:58.490881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 975/1934] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:14:58 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:14:58.503788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221f9cb-d3a9-4336-bb74-017651299a8c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:58.510780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2560a-8586-4f84-97e9-302a3555804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:58.512500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2560a-8586-4f84-97e9-302a3555804e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:58.513038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2560a-8586-4f84-97e9-302a3555804e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:58.513388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2560a-8586-4f84-97e9-302a3555804e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:58.514820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2560a-8586-4f84-97e9-302a3555804e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:58.518644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2560a-8586-4f84-97e9-302a3555804e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:58.541822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48142d29-9c48-41f4-a840-3db3a18ea16a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:58.542728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 960/1935] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:14:58.562374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48142d29-9c48-41f4-a840-3db3a18ea16a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:58.595883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485ed52-4dd4-46c2-af8c-166c0b68f087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:58.596936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485ed52-4dd4-46c2-af8c-166c0b68f087 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:58.597221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485ed52-4dd4-46c2-af8c-166c0b68f087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:58.597546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485ed52-4dd4-46c2-af8c-166c0b68f087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:58.598383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485ed52-4dd4-46c2-af8c-166c0b68f087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:58.600193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485ed52-4dd4-46c2-af8c-166c0b68f087 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:58.629092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 976/1936] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:14:58.634522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2560a-8586-4f84-97e9-302a3555804e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:58.662988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2560a-8586-4f84-97e9-302a3555804e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:58.670583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47f19e-376c-4c3a-b131-dffd6ef5cbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:58.671675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47f19e-376c-4c3a-b131-dffd6ef5cbf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:58.672105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47f19e-376c-4c3a-b131-dffd6ef5cbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:58.672462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47f19e-376c-4c3a-b131-dffd6ef5cbf1 admin admin] PATH_INFO: `/v3/projects/59a57b0c812f4af786ead221364ee1e1/users/7ae50bd429aa40ad89a262c5b80e2fac/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:58.695849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47f19e-376c-4c3a-b131-dffd6ef5cbf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a57b0c812f4af786ead221364ee1e1, user_id=7ae50bd429aa40ad89a262c5b80e2fac, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:58.698573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47f19e-376c-4c3a-b131-dffd6ef5cbf1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:58.720321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 961/1937] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:14:58 2022] PUT /identity/v3/projects/59a57b0c812f4af786ead221364ee1e1/users/7ae50bd429aa40ad89a262c5b80e2fac/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:58.732280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe05121-f925-4e0f-82f8-17cd3052a2cc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:58.732614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe05121-f925-4e0f-82f8-17cd3052a2cc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:58.732965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe05121-f925-4e0f-82f8-17cd3052a2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:58.772293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bbe05121-f925-4e0f-82f8-17cd3052a2cc None None] MFA Rules not processed for user `7ae50bd429aa40ad89a262c5b80e2fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:58.773635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bbe05121-f925-4e0f-82f8-17cd3052a2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:58.948078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47f19e-376c-4c3a-b131-dffd6ef5cbf1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:58.952194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bb47f19e-376c-4c3a-b131-dffd6ef5cbf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:58.980603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 977/1938] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:58.999893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe05121-f925-4e0f-82f8-17cd3052a2cc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:59.002601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bbe05121-f925-4e0f-82f8-17cd3052a2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:59.123363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe05121-f925-4e0f-82f8-17cd3052a2cc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:59.126279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47f19e-376c-4c3a-b131-dffd6ef5cbf1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:59.131232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00541f3b-61cd-4608-908c-61d75f767106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:59.133007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00541f3b-61cd-4608-908c-61d75f767106 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:59.133256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00541f3b-61cd-4608-908c-61d75f767106 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:59.133685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00541f3b-61cd-4608-908c-61d75f767106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:59.137267 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:59.139050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:59.140775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:59.142070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] PATH_INFO: `/v3/projects/f7dc2b8628704b68a6212cdbfe60cabc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:59.144605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7dc2b8628704b68a6212cdbfe60cabc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:59.148736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:59.153703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00541f3b-61cd-4608-908c-61d75f767106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:59.157733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00541f3b-61cd-4608-908c-61d75f767106 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:59.180366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 978/1939] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:14:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:14:59.260074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7dc2b8628704b68a6212cdbfe60cabc', 'request_id': 'req-9062d131-ec56-4659-b3c5-72a418ff9343'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:59.271735 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7dc2b8628704b68a6212cdbfe60cabc', 'request_id': 'req-9062d131-ec56-4659-b3c5-72a418ff9343'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:59.282084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7dc2b8628704b68a6212cdbfe60cabc', 'request_id': 'req-9062d131-ec56-4659-b3c5-72a418ff9343'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:59.302949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] The token cache is being invalidate because project f7dc2b8628704b68a6212cdbfe60cabc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:14:59.303298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:59.304106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9062d131-ec56-4659-b3c5-72a418ff9343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:14:59.305985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 962/1940] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:14:58 2022] DELETE /identity/v3/projects/f7dc2b8628704b68a6212cdbfe60cabc => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:14:59.314397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326ccdf-7555-4b6d-89c4-a497db7a0e03 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:59.314900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326ccdf-7555-4b6d-89c4-a497db7a0e03 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:59.315462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326ccdf-7555-4b6d-89c4-a497db7a0e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:59.346563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7326ccdf-7555-4b6d-89c4-a497db7a0e03 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:59.348783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7326ccdf-7555-4b6d-89c4-a497db7a0e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:59.578511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 979/1941] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:59.594161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06144-a9a0-417a-b087-f10ba5e84a8e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:59.595189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06144-a9a0-417a-b087-f10ba5e84a8e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:59.595846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06144-a9a0-417a-b087-f10ba5e84a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:59.623003 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:59.663482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c8f06144-a9a0-417a-b087-f10ba5e84a8e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:14:59.665544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8f06144-a9a0-417a-b087-f10ba5e84a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:14:59.699729 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326ccdf-7555-4b6d-89c4-a497db7a0e03 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:59.700615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 963/1942] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:14:59.775275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326ccdf-7555-4b6d-89c4-a497db7a0e03 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:59.818242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db6ef9-d020-425c-b13a-6cce59b8e094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:59.824942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db6ef9-d020-425c-b13a-6cce59b8e094 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:59.824942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db6ef9-d020-425c-b13a-6cce59b8e094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:59.824942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db6ef9-d020-425c-b13a-6cce59b8e094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:59.824942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db6ef9-d020-425c-b13a-6cce59b8e094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:14:59.824942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db6ef9-d020-425c-b13a-6cce59b8e094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:59.825554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db6ef9-d020-425c-b13a-6cce59b8e094 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:14:59.845746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:14:59.862471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f06144-a9a0-417a-b087-f10ba5e84a8e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:14:59.877038 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 980/1943] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:14:59.893372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f06144-a9a0-417a-b087-f10ba5e84a8e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:14:59.928009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb0957-df53-40e7-86e1-aed7006c70d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:14:59.929132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb0957-df53-40e7-86e1-aed7006c70d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:14:59.929801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb0957-df53-40e7-86e1-aed7006c70d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:14:59.930284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb0957-df53-40e7-86e1-aed7006c70d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:14:59.931470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb0957-df53-40e7-86e1-aed7006c70d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:14:59.934698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb0957-df53-40e7-86e1-aed7006c70d4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:00.014324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 964/1944] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:14:59 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:15:00.018086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db6ef9-d020-425c-b13a-6cce59b8e094 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:00.041725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db6ef9-d020-425c-b13a-6cce59b8e094 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:00.204073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da072c0f-415c-4f04-9f49-265a5c0b78a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:00.208056 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da072c0f-415c-4f04-9f49-265a5c0b78a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:00.209088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da072c0f-415c-4f04-9f49-265a5c0b78a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:00.209970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da072c0f-415c-4f04-9f49-265a5c0b78a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:00.211971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da072c0f-415c-4f04-9f49-265a5c0b78a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:00.217654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da072c0f-415c-4f04-9f49-265a5c0b78a0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:00.277966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb0957-df53-40e7-86e1-aed7006c70d4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:00.279549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 981/1945] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:00 2022] POST /identity/v3/users => generated 292 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:00.325284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb0957-df53-40e7-86e1-aed7006c70d4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:00.353134 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b72171-49fa-4505-ab9c-6bf3aa6e875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:00.354221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b72171-49fa-4505-ab9c-6bf3aa6e875a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:00.355454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b72171-49fa-4505-ab9c-6bf3aa6e875a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:00.357452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b72171-49fa-4505-ab9c-6bf3aa6e875a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:00.361348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b72171-49fa-4505-ab9c-6bf3aa6e875a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:00.375084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b72171-49fa-4505-ab9c-6bf3aa6e875a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:00.414317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 965/1946] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:00.420133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da072c0f-415c-4f04-9f49-265a5c0b78a0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:00.438994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da072c0f-415c-4f04-9f49-265a5c0b78a0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:00.444971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0169531-c5c3-4947-829a-143b58436f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:00.446053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0169531-c5c3-4947-829a-143b58436f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:00.446512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0169531-c5c3-4947-829a-143b58436f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:00.447150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0169531-c5c3-4947-829a-143b58436f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:00.448037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0169531-c5c3-4947-829a-143b58436f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:00.449960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0169531-c5c3-4947-829a-143b58436f2b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:00.463988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 982/1947] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:00.471357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b72171-49fa-4505-ab9c-6bf3aa6e875a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:00.493288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b72171-49fa-4505-ab9c-6bf3aa6e875a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:00.503927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77d52-6a00-47ae-abb8-ef99affcd122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:00.505980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77d52-6a00-47ae-abb8-ef99affcd122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:00.506928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77d52-6a00-47ae-abb8-ef99affcd122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:00.507734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77d52-6a00-47ae-abb8-ef99affcd122 admin admin] PATH_INFO: `/v3/projects/a8753595bdb9424b853be8525ed32007/users/2cde4b54d9484c198bf53a4ab6bf7deb/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:00.538360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a77d52-6a00-47ae-abb8-ef99affcd122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8753595bdb9424b853be8525ed32007, user_id=2cde4b54d9484c198bf53a4ab6bf7deb, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:00.543773 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a77d52-6a00-47ae-abb8-ef99affcd122 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:00.570921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 966/1948] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:00 2022] PUT /identity/v3/projects/a8753595bdb9424b853be8525ed32007/users/2cde4b54d9484c198bf53a4ab6bf7deb/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:00.585619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24302d-2319-432d-adfe-16f47c15156a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:00.586130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24302d-2319-432d-adfe-16f47c15156a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:00.586627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24302d-2319-432d-adfe-16f47c15156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:00.637865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ae24302d-2319-432d-adfe-16f47c15156a None None] MFA Rules not processed for user `2cde4b54d9484c198bf53a4ab6bf7deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:00.639342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ae24302d-2319-432d-adfe-16f47c15156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:00.738371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 983/1949] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:00.757206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77d52-6a00-47ae-abb8-ef99affcd122 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:00.759006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-85a77d52-6a00-47ae-abb8-ef99affcd122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:00.825143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77d52-6a00-47ae-abb8-ef99affcd122 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:00.829793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170e543-9780-4fa2-becc-69e7f1c73f13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:00.830501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e543-9780-4fa2-becc-69e7f1c73f13 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:00.830827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e543-9780-4fa2-becc-69e7f1c73f13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:00.831146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e543-9780-4fa2-becc-69e7f1c73f13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:00.841795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170e543-9780-4fa2-becc-69e7f1c73f13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:00.843514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170e543-9780-4fa2-becc-69e7f1c73f13 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:00.857551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 967/1950] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:00.886172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24302d-2319-432d-adfe-16f47c15156a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:01.006360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24302d-2319-432d-adfe-16f47c15156a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:01.013599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:01.014962 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:01.015446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:01.015933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] PATH_INFO: `/v3/users/2cde4b54d9484c198bf53a4ab6bf7deb` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:01.017940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cde4b54d9484c198bf53a4ab6bf7deb)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:01.021253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:01.041085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:01.121649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cde4b54d9484c198bf53a4ab6bf7deb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:01.122687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cde4b54d9484c198bf53a4ab6bf7deb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:01.137028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cde4b54d9484c198bf53a4ab6bf7deb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:01.169050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cde4b54d9484c198bf53a4ab6bf7deb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:01.187625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cde4b54d9484c198bf53a4ab6bf7deb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:01.199942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 984/1951] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:00 2022] DELETE /identity/v3/users/2cde4b54d9484c198bf53a4ab6bf7deb => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:01.207988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156254fb-f1ec-410c-abdd-f9f0591b073b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:01.208810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156254fb-f1ec-410c-abdd-f9f0591b073b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:01.208980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156254fb-f1ec-410c-abdd-f9f0591b073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:01.256507 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-156254fb-f1ec-410c-abdd-f9f0591b073b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:01.258965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-156254fb-f1ec-410c-abdd-f9f0591b073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:01.387231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 968/1952] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:01.408790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:01.412019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:01.528053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a795c-9c06-45e5-bea8-fe3023ac83ff admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:01.533636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e57bc9-a2f6-4116-afbd-658d28219b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:01.534387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e57bc9-a2f6-4116-afbd-658d28219b6e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:01.534820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e57bc9-a2f6-4116-afbd-658d28219b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:01.535314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e57bc9-a2f6-4116-afbd-658d28219b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:01.546607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e57bc9-a2f6-4116-afbd-658d28219b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:01.548361 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e57bc9-a2f6-4116-afbd-658d28219b6e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:01.560280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 985/1953] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:02.000887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156254fb-f1ec-410c-abdd-f9f0591b073b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:02.003280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-156254fb-f1ec-410c-abdd-f9f0591b073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:02.025880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156254fb-f1ec-410c-abdd-f9f0591b073b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:02.035120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:02.036699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:02.037214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:02.037906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] PATH_INFO: `/v3/projects/a8753595bdb9424b853be8525ed32007` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:02.040495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8753595bdb9424b853be8525ed32007)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:02.044232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:02.133880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:02.162184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8753595bdb9424b853be8525ed32007', 'request_id': 'req-3415a921-fdc4-4484-946c-cfa6f5932d3d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:02.183704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8753595bdb9424b853be8525ed32007', 'request_id': 'req-3415a921-fdc4-4484-946c-cfa6f5932d3d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:02.194111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8753595bdb9424b853be8525ed32007', 'request_id': 'req-3415a921-fdc4-4484-946c-cfa6f5932d3d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:02.216370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] The token cache is being invalidate because project a8753595bdb9424b853be8525ed32007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:02.217149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:02.217652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3415a921-fdc4-4484-946c-cfa6f5932d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:02.219655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 969/1954] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:01 2022] DELETE /identity/v3/projects/a8753595bdb9424b853be8525ed32007 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:02.228606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b24b4f-e2f7-41a2-b5d0-0b755ff145bb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:02.228959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b24b4f-e2f7-41a2-b5d0-0b755ff145bb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:02.229262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b24b4f-e2f7-41a2-b5d0-0b755ff145bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:02.258845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-01b24b4f-e2f7-41a2-b5d0-0b755ff145bb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:02.261072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-01b24b4f-e2f7-41a2-b5d0-0b755ff145bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:02.374292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 986/1955] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:02.388296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c490e78-fd5d-420a-a44a-ed6165cf93ff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:02.389163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c490e78-fd5d-420a-a44a-ed6165cf93ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:02.389651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c490e78-fd5d-420a-a44a-ed6165cf93ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:02.426260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4c490e78-fd5d-420a-a44a-ed6165cf93ff None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:02.428608 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4c490e78-fd5d-420a-a44a-ed6165cf93ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:02.444574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 970/1956] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:02.446142 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b24b4f-e2f7-41a2-b5d0-0b755ff145bb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:02.474243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b24b4f-e2f7-41a2-b5d0-0b755ff145bb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:02.496961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaf49d-b6fc-4855-98e7-d97970832a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:02.499104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaf49d-b6fc-4855-98e7-d97970832a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:02.508216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaf49d-b6fc-4855-98e7-d97970832a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:02.509943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaf49d-b6fc-4855-98e7-d97970832a83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:02.510948 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaf49d-b6fc-4855-98e7-d97970832a83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:02.511300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaf49d-b6fc-4855-98e7-d97970832a83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:02.512988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaf49d-b6fc-4855-98e7-d97970832a83 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:02.546116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 987/1957] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:15:02.550911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c490e78-fd5d-420a-a44a-ed6165cf93ff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:02.573393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c490e78-fd5d-420a-a44a-ed6165cf93ff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:02.593278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca3035-10c0-403a-ac70-00a455ed1a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:02.595057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca3035-10c0-403a-ac70-00a455ed1a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:02.608908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca3035-10c0-403a-ac70-00a455ed1a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:02.609563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca3035-10c0-403a-ac70-00a455ed1a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:02.610630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca3035-10c0-403a-ac70-00a455ed1a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:02.614786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca3035-10c0-403a-ac70-00a455ed1a23 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:02.671934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 971/1958] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:02 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:02.676501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaf49d-b6fc-4855-98e7-d97970832a83 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:02.687539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaf49d-b6fc-4855-98e7-d97970832a83 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:02.702032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:02.774709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded292df-c71a-4e48-855c-d53b6e76fcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:02.775915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded292df-c71a-4e48-855c-d53b6e76fcb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:02.776528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded292df-c71a-4e48-855c-d53b6e76fcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:02.777088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded292df-c71a-4e48-855c-d53b6e76fcb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:02.778382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded292df-c71a-4e48-855c-d53b6e76fcb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:02.782661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded292df-c71a-4e48-855c-d53b6e76fcb2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:02.819922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 988/1959] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:02 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:15:02.826348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca3035-10c0-403a-ac70-00a455ed1a23 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:02.846824 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca3035-10c0-403a-ac70-00a455ed1a23 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:02.856030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfdf95-6663-4f05-ade4-0d82cef52636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:02.857213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfdf95-6663-4f05-ade4-0d82cef52636 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:02.857705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfdf95-6663-4f05-ade4-0d82cef52636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:02.858185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfdf95-6663-4f05-ade4-0d82cef52636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:02.859615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfdf95-6663-4f05-ade4-0d82cef52636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:02.863727 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfdf95-6663-4f05-ade4-0d82cef52636 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:02.882662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 972/1960] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:02.884374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded292df-c71a-4e48-855c-d53b6e76fcb2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:02.900109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded292df-c71a-4e48-855c-d53b6e76fcb2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:02.905271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15638b05-2053-493b-a9da-5bc5dd012678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:02.906173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15638b05-2053-493b-a9da-5bc5dd012678 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:02.906461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15638b05-2053-493b-a9da-5bc5dd012678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:02.907281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15638b05-2053-493b-a9da-5bc5dd012678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:02.908053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15638b05-2053-493b-a9da-5bc5dd012678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:02.909691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15638b05-2053-493b-a9da-5bc5dd012678 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:02.928469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 989/1961] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:02.929816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfdf95-6663-4f05-ade4-0d82cef52636 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:02.941386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfdf95-6663-4f05-ade4-0d82cef52636 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:02.951737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7bf3d-6976-4be5-9fea-3f9681af24bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:02.953408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7bf3d-6976-4be5-9fea-3f9681af24bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:02.954458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7bf3d-6976-4be5-9fea-3f9681af24bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:02.954967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7bf3d-6976-4be5-9fea-3f9681af24bc admin admin] PATH_INFO: `/v3/projects/48627b1b45b7421788fff8895b01a12f/users/d08eab52d8b24df4b7b3fbad709c48bf/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:02.975707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7bf3d-6976-4be5-9fea-3f9681af24bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=48627b1b45b7421788fff8895b01a12f, user_id=d08eab52d8b24df4b7b3fbad709c48bf, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:02.978437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7bf3d-6976-4be5-9fea-3f9681af24bc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:03.000415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 973/1962] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:02 2022] PUT /identity/v3/projects/48627b1b45b7421788fff8895b01a12f/users/d08eab52d8b24df4b7b3fbad709c48bf/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:03.021394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b466b2b-2daa-491f-804b-d9557552d1b9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:03.022508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b466b2b-2daa-491f-804b-d9557552d1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:03.023469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b466b2b-2daa-491f-804b-d9557552d1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:03.073765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5b466b2b-2daa-491f-804b-d9557552d1b9 None None] MFA Rules not processed for user `d08eab52d8b24df4b7b3fbad709c48bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:03.075324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5b466b2b-2daa-491f-804b-d9557552d1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:03.197434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 990/1963] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:03.210363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7bf3d-6976-4be5-9fea-3f9681af24bc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:03.213206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-39a7bf3d-6976-4be5-9fea-3f9681af24bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:03.328362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7bf3d-6976-4be5-9fea-3f9681af24bc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:03.333655 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be5431-1327-4d80-9e47-224ce0581448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:03.334456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be5431-1327-4d80-9e47-224ce0581448 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:03.334842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be5431-1327-4d80-9e47-224ce0581448 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:03.335221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be5431-1327-4d80-9e47-224ce0581448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:03.347368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be5431-1327-4d80-9e47-224ce0581448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:03.349629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be5431-1327-4d80-9e47-224ce0581448 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:03.363770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 974/1964] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:12.462091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b466b2b-2daa-491f-804b-d9557552d1b9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:12.463634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5b466b2b-2daa-491f-804b-d9557552d1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:12.571000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b466b2b-2daa-491f-804b-d9557552d1b9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:12.577046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:12.578146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:12.579179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:12.579557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] PATH_INFO: `/v3/users/29fd5aa8f0ef479d9c7ff20dbb503e87` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:12.581269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29fd5aa8f0ef479d9c7ff20dbb503e87)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:12.584166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:12.631294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:12.660196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29fd5aa8f0ef479d9c7ff20dbb503e87'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:12.661334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29fd5aa8f0ef479d9c7ff20dbb503e87'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:12.674610 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29fd5aa8f0ef479d9c7ff20dbb503e87'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:12.696074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29fd5aa8f0ef479d9c7ff20dbb503e87'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:12.709921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29fd5aa8f0ef479d9c7ff20dbb503e87'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:12.721363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 991/1965] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:12 2022] DELETE /identity/v3/users/29fd5aa8f0ef479d9c7ff20dbb503e87 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:15:12.728082 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9363546-ef14-459e-ab1f-96c182ec3795 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:12.729809 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9363546-ef14-459e-ab1f-96c182ec3795 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:12.730727 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9363546-ef14-459e-ab1f-96c182ec3795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:12.757842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b9363546-ef14-459e-ab1f-96c182ec3795 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:12.758880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b9363546-ef14-459e-ab1f-96c182ec3795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:12.864472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 975/1966] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:12.881159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:12.883124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:12.983896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0c80a-cc71-4b1b-82fe-d9c8ae96c621 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:12.992107 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66edc9d-481c-4864-a1ff-2c68512e0fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:12.993374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66edc9d-481c-4864-a1ff-2c68512e0fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:12.993989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66edc9d-481c-4864-a1ff-2c68512e0fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:12.994589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66edc9d-481c-4864-a1ff-2c68512e0fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:13.040044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66edc9d-481c-4864-a1ff-2c68512e0fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:13.045538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66edc9d-481c-4864-a1ff-2c68512e0fd0 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:13.066446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 992/1967] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:15:13.381745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9363546-ef14-459e-ab1f-96c182ec3795 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:13.384609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b9363546-ef14-459e-ab1f-96c182ec3795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:13.421035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9363546-ef14-459e-ab1f-96c182ec3795 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:13.429509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:13.431322 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:13.431996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:13.432649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] PATH_INFO: `/v3/users/d08eab52d8b24df4b7b3fbad709c48bf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:13.435543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08eab52d8b24df4b7b3fbad709c48bf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:13.440400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:13.452999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:13.587295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08eab52d8b24df4b7b3fbad709c48bf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:13.614718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08eab52d8b24df4b7b3fbad709c48bf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:13.646179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08eab52d8b24df4b7b3fbad709c48bf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:13.695980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08eab52d8b24df4b7b3fbad709c48bf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:13.698847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08eab52d8b24df4b7b3fbad709c48bf'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:13.723425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc48fae-2cd1-4bf1-be75-ae0819efc1bb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:13.724110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc48fae-2cd1-4bf1-be75-ae0819efc1bb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:13.724844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc48fae-2cd1-4bf1-be75-ae0819efc1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:13.728352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 976/1968] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:13 2022] DELETE /identity/v3/users/d08eab52d8b24df4b7b3fbad709c48bf => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:13.731148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:13.737010 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:13.761584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2bc48fae-2cd1-4bf1-be75-ae0819efc1bb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:13.762804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2bc48fae-2cd1-4bf1-be75-ae0819efc1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:13.885348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 993/1969] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:13.902924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc48fae-2cd1-4bf1-be75-ae0819efc1bb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:13.905641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2bc48fae-2cd1-4bf1-be75-ae0819efc1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:13.911880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68048ff-39b3-478d-b46b-d2a552acd2ae admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:13.923127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:13.925183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:13.925710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:13.926223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] PATH_INFO: `/v3/projects/c54daeb26bc247049e61b4364036be0a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:13.929484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c54daeb26bc247049e61b4364036be0a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:13.933520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:14.016095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:14.025489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc48fae-2cd1-4bf1-be75-ae0819efc1bb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:14.032974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c54daeb26bc247049e61b4364036be0a', 'request_id': 'req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.045449 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c54daeb26bc247049e61b4364036be0a', 'request_id': 'req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.063756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c54daeb26bc247049e61b4364036be0a', 'request_id': 'req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.089335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] The token cache is being invalidate because project c54daeb26bc247049e61b4364036be0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:14.089335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.089803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.092600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 977/1970] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:13 2022] DELETE /identity/v3/projects/c54daeb26bc247049e61b4364036be0a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:14.097905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:14.100022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:14.116756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:14.121189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:14.121930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:14.122241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:14.123380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:14.125579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:14.217238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b22be-cf0f-4a91-b88f-3f60d8b096c1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:14.224006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:14.225046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:14.225343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:14.225652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] PATH_INFO: `/v3/users/f8f05a002254415ea0ac81902635ed6e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:14.228030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f05a002254415ea0ac81902635ed6e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:14.229997 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:14.248298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:14.254997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:14.271711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 994/1971] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:14.303395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f05a002254415ea0ac81902635ed6e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.322466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f05a002254415ea0ac81902635ed6e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.341891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f05a002254415ea0ac81902635ed6e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.366931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f05a002254415ea0ac81902635ed6e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.370380 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f05a002254415ea0ac81902635ed6e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:14.384691 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 978/1972] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:14 2022] DELETE /identity/v3/users/f8f05a002254415ea0ac81902635ed6e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:15:14.933337 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:14.935049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:14.952759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fe603-4a92-47ba-963d-a675ed1ea3b9 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:15.080205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:15.082270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:15.082270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:15.082699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] PATH_INFO: `/v3/projects/48627b1b45b7421788fff8895b01a12f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:15.084293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48627b1b45b7421788fff8895b01a12f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:15.086783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:15.143947 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:15.148274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:15.197159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667f657-6240-45ce-8994-a30ec62448b6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:15.216867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48627b1b45b7421788fff8895b01a12f', 'request_id': 'req-e0dfd350-f306-4e08-a522-3c39a09fece9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.227607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:15.230154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48627b1b45b7421788fff8895b01a12f', 'request_id': 'req-e0dfd350-f306-4e08-a522-3c39a09fece9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.246969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48627b1b45b7421788fff8895b01a12f', 'request_id': 'req-e0dfd350-f306-4e08-a522-3c39a09fece9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.270176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] The token cache is being invalidate because project 48627b1b45b7421788fff8895b01a12f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:15.270775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.271698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e0dfd350-f306-4e08-a522-3c39a09fece9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.273934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 995/1973] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:14 2022] DELETE /identity/v3/projects/48627b1b45b7421788fff8895b01a12f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:15.362517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:15.364464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:15.364762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:15.365051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] PATH_INFO: `/v3/projects/ff247488a6914b45a0879bf4420d35fd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:15.370102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff247488a6914b45a0879bf4420d35fd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:15.372386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:15.446462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:15.473162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff247488a6914b45a0879bf4420d35fd', 'request_id': 'req-fa093ff5-802d-48bc-ba0e-0f897383d81c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.493536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff247488a6914b45a0879bf4420d35fd', 'request_id': 'req-fa093ff5-802d-48bc-ba0e-0f897383d81c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.511121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff247488a6914b45a0879bf4420d35fd', 'request_id': 'req-fa093ff5-802d-48bc-ba0e-0f897383d81c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.554509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] The token cache is being invalidate because project ff247488a6914b45a0879bf4420d35fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:15.554509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.554509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa093ff5-802d-48bc-ba0e-0f897383d81c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:15.554967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 979/1974] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:15 2022] DELETE /identity/v3/projects/ff247488a6914b45a0879bf4420d35fd => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:15:15.564949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e7f2b-db40-40b5-9a58-b23c5a723dd7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:15.564949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e7f2b-db40-40b5-9a58-b23c5a723dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:15.565299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e7f2b-db40-40b5-9a58-b23c5a723dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:15.602606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8d6e7f2b-db40-40b5-9a58-b23c5a723dd7 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:15.605181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8d6e7f2b-db40-40b5-9a58-b23c5a723dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:15.686266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 996/1975] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:15.694841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b06f6-660f-4fe4-b687-f1ee5123e030 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:15.695151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b06f6-660f-4fe4-b687-f1ee5123e030 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:15.695637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b06f6-660f-4fe4-b687-f1ee5123e030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:15.730879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3e3b06f6-660f-4fe4-b687-f1ee5123e030 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:15.736728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3e3b06f6-660f-4fe4-b687-f1ee5123e030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:15.753179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 980/1976] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:15:15.759399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e7f2b-db40-40b5-9a58-b23c5a723dd7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:15.777698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e7f2b-db40-40b5-9a58-b23c5a723dd7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:15.794296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77a8a-65e7-4be2-9bd5-d5c6c33268c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:15.795626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77a8a-65e7-4be2-9bd5-d5c6c33268c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:15.796217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77a8a-65e7-4be2-9bd5-d5c6c33268c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:15.796794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77a8a-65e7-4be2-9bd5-d5c6c33268c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:15.798160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c77a8a-65e7-4be2-9bd5-d5c6c33268c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:15.798801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c77a8a-65e7-4be2-9bd5-d5c6c33268c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:15.801993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c77a8a-65e7-4be2-9bd5-d5c6c33268c7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:15.831419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 997/1977] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:15:15.832535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b06f6-660f-4fe4-b687-f1ee5123e030 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:15.858192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b06f6-660f-4fe4-b687-f1ee5123e030 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:15.867411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4656d-e526-4bc2-9023-b7f46411e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:15.868214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4656d-e526-4bc2-9023-b7f46411e619 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:15.868489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4656d-e526-4bc2-9023-b7f46411e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:15.868764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4656d-e526-4bc2-9023-b7f46411e619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:15.869465 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4656d-e526-4bc2-9023-b7f46411e619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:15.871452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4656d-e526-4bc2-9023-b7f46411e619 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:15.913637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 981/1978] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:15 2022] POST /identity/v3/projects => generated 355 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:15.918406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77a8a-65e7-4be2-9bd5-d5c6c33268c7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:15.929188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77a8a-65e7-4be2-9bd5-d5c6c33268c7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:16.025962 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cb476-1295-43b6-b28f-80a651dfff3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:16.027055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cb476-1295-43b6-b28f-80a651dfff3e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:16.027739 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cb476-1295-43b6-b28f-80a651dfff3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:16.028585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cb476-1295-43b6-b28f-80a651dfff3e admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:16.029879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cb476-1295-43b6-b28f-80a651dfff3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:16.035011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cb476-1295-43b6-b28f-80a651dfff3e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:16.070831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 998/1979] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:15 2022] POST /identity/v3/users => generated 278 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:16.072666 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4656d-e526-4bc2-9023-b7f46411e619 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:16.087615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4656d-e526-4bc2-9023-b7f46411e619 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:16.094663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9591cc-8ca9-42a6-9d3b-c4f2ffa249bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:16.095643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9591cc-8ca9-42a6-9d3b-c4f2ffa249bb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:16.096026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9591cc-8ca9-42a6-9d3b-c4f2ffa249bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:16.096411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9591cc-8ca9-42a6-9d3b-c4f2ffa249bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:16.097698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9591cc-8ca9-42a6-9d3b-c4f2ffa249bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:16.100201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9591cc-8ca9-42a6-9d3b-c4f2ffa249bb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:16.119295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 982/1980] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:15:16.125720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cb476-1295-43b6-b28f-80a651dfff3e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:16.140217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cb476-1295-43b6-b28f-80a651dfff3e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:16.151306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79774f99-5b2f-4659-a8c3-87c49a3b7737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:16.152373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79774f99-5b2f-4659-a8c3-87c49a3b7737 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:16.152624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79774f99-5b2f-4659-a8c3-87c49a3b7737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:16.153021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79774f99-5b2f-4659-a8c3-87c49a3b7737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:16.154055 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79774f99-5b2f-4659-a8c3-87c49a3b7737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:16.155913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79774f99-5b2f-4659-a8c3-87c49a3b7737 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:16.169009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 999/1981] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:16.173352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9591cc-8ca9-42a6-9d3b-c4f2ffa249bb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:16.184243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9591cc-8ca9-42a6-9d3b-c4f2ffa249bb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:16.194141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a9d41-5084-4d17-901c-1a77e6ff3dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:16.195807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a9d41-5084-4d17-901c-1a77e6ff3dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:16.196286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a9d41-5084-4d17-901c-1a77e6ff3dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:16.197636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a9d41-5084-4d17-901c-1a77e6ff3dde admin admin] PATH_INFO: `/v3/projects/2cb7e1b8314d4b3e9eb1d851f3efc5ca/users/fe8c83a278d24dd3bcdc0a80379e0ce9/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:16.232095 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a9d41-5084-4d17-901c-1a77e6ff3dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb7e1b8314d4b3e9eb1d851f3efc5ca, user_id=fe8c83a278d24dd3bcdc0a80379e0ce9, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:16.238779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a9d41-5084-4d17-901c-1a77e6ff3dde admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:16.267669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 983/1982] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:16 2022] PUT /identity/v3/projects/2cb7e1b8314d4b3e9eb1d851f3efc5ca/users/fe8c83a278d24dd3bcdc0a80379e0ce9/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:15:16.282689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418571c7-d1c1-4428-89f6-7f0da0c0c159 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:16.283174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418571c7-d1c1-4428-89f6-7f0da0c0c159 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:16.283694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418571c7-d1c1-4428-89f6-7f0da0c0c159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:16.324014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-418571c7-d1c1-4428-89f6-7f0da0c0c159 None None] MFA Rules not processed for user `fe8c83a278d24dd3bcdc0a80379e0ce9`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:16.325355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-418571c7-d1c1-4428-89f6-7f0da0c0c159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:16.390369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:16.436101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1000/1983] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:16 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:16.442021 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a9d41-5084-4d17-901c-1a77e6ff3dde admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:16.553765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a9d41-5084-4d17-901c-1a77e6ff3dde admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:16.560707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4d6d8-aab5-478a-9629-6912e46ca6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:16.561539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4d6d8-aab5-478a-9629-6912e46ca6ed admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:16.561828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4d6d8-aab5-478a-9629-6912e46ca6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:16.562313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4d6d8-aab5-478a-9629-6912e46ca6ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:16.564210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4d6d8-aab5-478a-9629-6912e46ca6ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:16.566253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4d6d8-aab5-478a-9629-6912e46ca6ed admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:16.603128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 984/1984] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:16 2022] POST /identity/v3/projects => generated 355 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:16.607533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418571c7-d1c1-4428-89f6-7f0da0c0c159 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:16.623135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418571c7-d1c1-4428-89f6-7f0da0c0c159 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:16.700659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bee70a-ea51-4460-8be1-f32911b09ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:16.701374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bee70a-ea51-4460-8be1-f32911b09ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:16.701674 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bee70a-ea51-4460-8be1-f32911b09ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:16.701979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bee70a-ea51-4460-8be1-f32911b09ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:16.702718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bee70a-ea51-4460-8be1-f32911b09ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:16.704871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bee70a-ea51-4460-8be1-f32911b09ae7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:16.733568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1001/1985] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:16 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:15:16.735536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4d6d8-aab5-478a-9629-6912e46ca6ed admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:16.749261 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4d6d8-aab5-478a-9629-6912e46ca6ed admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:16.755092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a8cc7-81a8-4741-b953-bbfc7c68bfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:16.755909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a8cc7-81a8-4741-b953-bbfc7c68bfda admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:16.756346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a8cc7-81a8-4741-b953-bbfc7c68bfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:16.756662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a8cc7-81a8-4741-b953-bbfc7c68bfda admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:16.757426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a8cc7-81a8-4741-b953-bbfc7c68bfda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:16.759099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a8cc7-81a8-4741-b953-bbfc7c68bfda admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:16.772661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 985/1986] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:16.778375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bee70a-ea51-4460-8be1-f32911b09ae7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:16.796899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bee70a-ea51-4460-8be1-f32911b09ae7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:16.804899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef113e3-f586-4457-9382-279c5a35e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:16.806607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef113e3-f586-4457-9382-279c5a35e882 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:16.807121 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef113e3-f586-4457-9382-279c5a35e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:16.807687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef113e3-f586-4457-9382-279c5a35e882 admin admin] PATH_INFO: `/v3/projects/4c315501c78148cf973f8f9b732b86ab/users/33c4502b2643485fa8bb562d4f901370/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:16.834127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef113e3-f586-4457-9382-279c5a35e882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c315501c78148cf973f8f9b732b86ab, user_id=33c4502b2643485fa8bb562d4f901370, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:16.837567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef113e3-f586-4457-9382-279c5a35e882 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:16.871856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1002/1987] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:16 2022] PUT /identity/v3/projects/4c315501c78148cf973f8f9b732b86ab/users/33c4502b2643485fa8bb562d4f901370/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:16.878569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a8cc7-81a8-4741-b953-bbfc7c68bfda admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:16.917779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a8cc7-81a8-4741-b953-bbfc7c68bfda admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:16.966199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:17.012180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fba3a-7abd-4de1-bf18-7a06d618cd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:17.012872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fba3a-7abd-4de1-bf18-7a06d618cd56 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:17.013159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fba3a-7abd-4de1-bf18-7a06d618cd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:17.013436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fba3a-7abd-4de1-bf18-7a06d618cd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:17.014174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fba3a-7abd-4de1-bf18-7a06d618cd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:17.015932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fba3a-7abd-4de1-bf18-7a06d618cd56 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:17.027684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 986/1988] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:17.033796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef113e3-f586-4457-9382-279c5a35e882 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:17.048467 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef113e3-f586-4457-9382-279c5a35e882 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:17.056446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc5e70-0f99-4f13-b94c-c9d3cca60cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:17.058360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc5e70-0f99-4f13-b94c-c9d3cca60cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:17.058849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc5e70-0f99-4f13-b94c-c9d3cca60cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:17.059327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc5e70-0f99-4f13-b94c-c9d3cca60cc3 admin admin] PATH_INFO: `/v3/projects/4c315501c78148cf973f8f9b732b86ab/users/33c4502b2643485fa8bb562d4f901370/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:17.062967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc5e70-0f99-4f13-b94c-c9d3cca60cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c315501c78148cf973f8f9b732b86ab, user_id=33c4502b2643485fa8bb562d4f901370, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:17.067615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc5e70-0f99-4f13-b94c-c9d3cca60cc3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:17.085491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1003/1989] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:17 2022] PUT /identity/v3/projects/4c315501c78148cf973f8f9b732b86ab/users/33c4502b2643485fa8bb562d4f901370/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:15:17.105203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175ae21-4440-49a8-bbcc-576a8fa0ff43 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:17.105833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175ae21-4440-49a8-bbcc-576a8fa0ff43 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:17.106613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175ae21-4440-49a8-bbcc-576a8fa0ff43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:17.155159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5175ae21-4440-49a8-bbcc-576a8fa0ff43 None None] MFA Rules not processed for user `33c4502b2643485fa8bb562d4f901370`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:17.156909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5175ae21-4440-49a8-bbcc-576a8fa0ff43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:17.262602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 987/1990] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:17 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:17.271552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc5e70-0f99-4f13-b94c-c9d3cca60cc3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:17.273584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-79fc5e70-0f99-4f13-b94c-c9d3cca60cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:17.342824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc5e70-0f99-4f13-b94c-c9d3cca60cc3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:17.347707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a2664-f124-4563-ba50-3cec851d029b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:17.348519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a2664-f124-4563-ba50-3cec851d029b service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:17.348977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a2664-f124-4563-ba50-3cec851d029b service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:17.349302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a2664-f124-4563-ba50-3cec851d029b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:17.362640 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a2664-f124-4563-ba50-3cec851d029b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:17.365032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a2664-f124-4563-ba50-3cec851d029b service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:17.379489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1004/1991] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:17 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:17.488732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175ae21-4440-49a8-bbcc-576a8fa0ff43 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:17.491399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5175ae21-4440-49a8-bbcc-576a8fa0ff43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:17.514053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175ae21-4440-49a8-bbcc-576a8fa0ff43 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:17.519872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed6a4b-7a91-4e65-a22a-b8b12f6125e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:17.520903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed6a4b-7a91-4e65-a22a-b8b12f6125e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:17.521271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed6a4b-7a91-4e65-a22a-b8b12f6125e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:17.521673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed6a4b-7a91-4e65-a22a-b8b12f6125e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:17.634598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed6a4b-7a91-4e65-a22a-b8b12f6125e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:17.636145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed6a4b-7a91-4e65-a22a-b8b12f6125e2 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:17.657527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 988/1992] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:17 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:19.974231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a2664-f124-4563-ba50-3cec851d029b service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:19.976282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-af4a2664-f124-4563-ba50-3cec851d029b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:20.031723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:20.079053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a2664-f124-4563-ba50-3cec851d029b service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:20.086830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:20.088465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:20.088958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:20.089407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] PATH_INFO: `/v3/users/7ae50bd429aa40ad89a262c5b80e2fac` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:20.091243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae50bd429aa40ad89a262c5b80e2fac)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:20.094262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:20.182421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae50bd429aa40ad89a262c5b80e2fac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:20.183517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae50bd429aa40ad89a262c5b80e2fac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:20.197013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae50bd429aa40ad89a262c5b80e2fac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:20.220129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae50bd429aa40ad89a262c5b80e2fac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:20.231065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae50bd429aa40ad89a262c5b80e2fac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:20.243436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1005/1993] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:19 2022] DELETE /identity/v3/users/7ae50bd429aa40ad89a262c5b80e2fac => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:20.249952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43621769-7d41-4cb9-86bb-f2662f9f2b2e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:20.250763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43621769-7d41-4cb9-86bb-f2662f9f2b2e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:20.250763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43621769-7d41-4cb9-86bb-f2662f9f2b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:20.269467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-43621769-7d41-4cb9-86bb-f2662f9f2b2e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:20.270781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-43621769-7d41-4cb9-86bb-f2662f9f2b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:20.336846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:20.363838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 989/1994] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:15:20.379318 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:20.381943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:20.473602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011d8b1-0848-440a-88d5-accc5363e276 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:20.478546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b9a0-04ea-422e-a9db-b431dda9a39b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:20.479682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b9a0-04ea-422e-a9db-b431dda9a39b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:20.479682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b9a0-04ea-422e-a9db-b431dda9a39b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:20.480159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b9a0-04ea-422e-a9db-b431dda9a39b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:20.501235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b9a0-04ea-422e-a9db-b431dda9a39b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:20.503731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b9a0-04ea-422e-a9db-b431dda9a39b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:20.519168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1006/1995] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:15:21.086221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43621769-7d41-4cb9-86bb-f2662f9f2b2e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:21.087770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-43621769-7d41-4cb9-86bb-f2662f9f2b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:21.106100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43621769-7d41-4cb9-86bb-f2662f9f2b2e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:21.111571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:21.112744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.113132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.113617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] PATH_INFO: `/v3/projects/59a57b0c812f4af786ead221364ee1e1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.115642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a57b0c812f4af786ead221364ee1e1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:21.118605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:21.179253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a57b0c812f4af786ead221364ee1e1', 'request_id': 'req-da0c8602-3414-4807-8c9a-d8588b90642d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:21.186001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a57b0c812f4af786ead221364ee1e1', 'request_id': 'req-da0c8602-3414-4807-8c9a-d8588b90642d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:21.194206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a57b0c812f4af786ead221364ee1e1', 'request_id': 'req-da0c8602-3414-4807-8c9a-d8588b90642d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:21.209975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] The token cache is being invalidate because project 59a57b0c812f4af786ead221364ee1e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:21.210274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:21.211029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-da0c8602-3414-4807-8c9a-d8588b90642d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:21.212963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 990/1996] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:21 2022] DELETE /identity/v3/projects/59a57b0c812f4af786ead221364ee1e1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:21.218539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17698b3f-6ccd-4188-bc61-306705145fd9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.219341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17698b3f-6ccd-4188-bc61-306705145fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.219535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17698b3f-6ccd-4188-bc61-306705145fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.249248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-17698b3f-6ccd-4188-bc61-306705145fd9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:21.251533 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-17698b3f-6ccd-4188-bc61-306705145fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:21.320986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:21.336123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1007/1997] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:21.345247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556f5e5-7fc7-4464-9ce7-1dc76b60bd26 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.345910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556f5e5-7fc7-4464-9ce7-1dc76b60bd26 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.345910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556f5e5-7fc7-4464-9ce7-1dc76b60bd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.366005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9556f5e5-7fc7-4464-9ce7-1dc76b60bd26 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:21.367417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9556f5e5-7fc7-4464-9ce7-1dc76b60bd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:21.382090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 991/1998] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:15:21.384408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17698b3f-6ccd-4188-bc61-306705145fd9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:21.399605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17698b3f-6ccd-4188-bc61-306705145fd9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:21.406554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02013620-4cff-4cb6-bc22-37819e8712e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:21.407332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013620-4cff-4cb6-bc22-37819e8712e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.407629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013620-4cff-4cb6-bc22-37819e8712e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.407912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013620-4cff-4cb6-bc22-37819e8712e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.408664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02013620-4cff-4cb6-bc22-37819e8712e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:21.408966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02013620-4cff-4cb6-bc22-37819e8712e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:21.410596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02013620-4cff-4cb6-bc22-37819e8712e6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:21.438487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1008/1999] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:15:21.439782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556f5e5-7fc7-4464-9ce7-1dc76b60bd26 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:21.451395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556f5e5-7fc7-4464-9ce7-1dc76b60bd26 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:21.461265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8c3bc-38b9-4b73-a464-6bad78051817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:21.462437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8c3bc-38b9-4b73-a464-6bad78051817 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.462925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8c3bc-38b9-4b73-a464-6bad78051817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.463474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8c3bc-38b9-4b73-a464-6bad78051817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.464616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8c3bc-38b9-4b73-a464-6bad78051817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:21.467800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8c3bc-38b9-4b73-a464-6bad78051817 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:21.511434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 992/2000] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:21 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:21.515280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02013620-4cff-4cb6-bc22-37819e8712e6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:21.530107 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02013620-4cff-4cb6-bc22-37819e8712e6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:21.628969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbd76a-6e42-42b4-8381-80e573c39e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:21.629660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbd76a-6e42-42b4-8381-80e573c39e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.629953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbd76a-6e42-42b4-8381-80e573c39e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.630235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbd76a-6e42-42b4-8381-80e573c39e80 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.630955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbd76a-6e42-42b4-8381-80e573c39e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:21.632936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbd76a-6e42-42b4-8381-80e573c39e80 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:21.659716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1009/2001] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:21 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:21.664702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8c3bc-38b9-4b73-a464-6bad78051817 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:21.677238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8c3bc-38b9-4b73-a464-6bad78051817 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:21.682221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783f81c-4d19-477e-90f5-c3149e2ae4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:21.682895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783f81c-4d19-477e-90f5-c3149e2ae4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.683186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783f81c-4d19-477e-90f5-c3149e2ae4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.683501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783f81c-4d19-477e-90f5-c3149e2ae4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.684237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783f81c-4d19-477e-90f5-c3149e2ae4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:21.685859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783f81c-4d19-477e-90f5-c3149e2ae4b6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:21.701503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 993/2002] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:15:21.707665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbd76a-6e42-42b4-8381-80e573c39e80 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:21.719190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbd76a-6e42-42b4-8381-80e573c39e80 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:21.724078 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba21ba-c4cf-4875-a98c-3643b1164d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:21.724753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba21ba-c4cf-4875-a98c-3643b1164d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.725041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba21ba-c4cf-4875-a98c-3643b1164d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.725319 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba21ba-c4cf-4875-a98c-3643b1164d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.726050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba21ba-c4cf-4875-a98c-3643b1164d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:21.727685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba21ba-c4cf-4875-a98c-3643b1164d5d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:21.741766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1010/2003] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:15:21.746402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783f81c-4d19-477e-90f5-c3149e2ae4b6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:21.758631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783f81c-4d19-477e-90f5-c3149e2ae4b6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:21.763940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6e650-585a-4a74-b9e9-6ac19c8969ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:21.765115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6e650-585a-4a74-b9e9-6ac19c8969ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.766359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6e650-585a-4a74-b9e9-6ac19c8969ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.766901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6e650-585a-4a74-b9e9-6ac19c8969ac admin admin] PATH_INFO: `/v3/projects/a84fa2f397c54be1b409d8be5ebc8491/users/e908d2b09dfc4a139d9c27f52580e15b/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.793638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6e650-585a-4a74-b9e9-6ac19c8969ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84fa2f397c54be1b409d8be5ebc8491, user_id=e908d2b09dfc4a139d9c27f52580e15b, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:21.799114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6e650-585a-4a74-b9e9-6ac19c8969ac admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:21.824931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 994/2004] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:21 2022] PUT /identity/v3/projects/a84fa2f397c54be1b409d8be5ebc8491/users/e908d2b09dfc4a139d9c27f52580e15b/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:21.834528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c907c2-8cd4-4a77-8d6b-401e6b66b335 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:21.835018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c907c2-8cd4-4a77-8d6b-401e6b66b335 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:21.835637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c907c2-8cd4-4a77-8d6b-401e6b66b335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:21.867898 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-22c907c2-8cd4-4a77-8d6b-401e6b66b335 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:21.870726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-22c907c2-8cd4-4a77-8d6b-401e6b66b335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:22.024510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1011/2005] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:22.034893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6e650-585a-4a74-b9e9-6ac19c8969ac admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:22.037563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-42a6e650-585a-4a74-b9e9-6ac19c8969ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:22.264454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6e650-585a-4a74-b9e9-6ac19c8969ac admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:22.287653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3aaa00-dff6-48b2-8d3b-9c7cb474dc1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:22.289087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3aaa00-dff6-48b2-8d3b-9c7cb474dc1d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:22.289692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3aaa00-dff6-48b2-8d3b-9c7cb474dc1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:22.290258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3aaa00-dff6-48b2-8d3b-9c7cb474dc1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:22.316633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3aaa00-dff6-48b2-8d3b-9c7cb474dc1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:22.321039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3aaa00-dff6-48b2-8d3b-9c7cb474dc1d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:22.338992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 995/2006] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:29.327779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008ed5-b42c-43db-9077-b0c01c340d95 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:29.329331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008ed5-b42c-43db-9077-b0c01c340d95 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:29.329946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008ed5-b42c-43db-9077-b0c01c340d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:29.376617 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-41008ed5-b42c-43db-9077-b0c01c340d95 None None] MFA Rules not processed for user `e908d2b09dfc4a139d9c27f52580e15b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:29.379608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-41008ed5-b42c-43db-9077-b0c01c340d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:29.503563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:29.536251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1012/2007] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:15:29.541823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3aaa00-dff6-48b2-8d3b-9c7cb474dc1d service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:29.555645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3aaa00-dff6-48b2-8d3b-9c7cb474dc1d service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:29.568561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8561ce-717b-4bf2-bd50-a3f41a0843a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:29.569786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8561ce-717b-4bf2-bd50-a3f41a0843a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:29.570671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8561ce-717b-4bf2-bd50-a3f41a0843a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:29.571694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8561ce-717b-4bf2-bd50-a3f41a0843a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:29.572958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8561ce-717b-4bf2-bd50-a3f41a0843a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:29.577426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8561ce-717b-4bf2-bd50-a3f41a0843a5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:29.637152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:29.679035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 996/2008] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:29 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:15:29.688826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41008ed5-b42c-43db-9077-b0c01c340d95 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:29.706569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41008ed5-b42c-43db-9077-b0c01c340d95 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:29.847928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01afe1-ed93-47a1-8901-2a17a69334e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:29.848354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01afe1-ed93-47a1-8901-2a17a69334e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:29.848654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01afe1-ed93-47a1-8901-2a17a69334e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:29.849054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01afe1-ed93-47a1-8901-2a17a69334e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:29.849858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01afe1-ed93-47a1-8901-2a17a69334e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:29.852447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01afe1-ed93-47a1-8901-2a17a69334e3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:29.900112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1013/2009] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:29 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:29.901899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8561ce-717b-4bf2-bd50-a3f41a0843a5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:29.926827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8561ce-717b-4bf2-bd50-a3f41a0843a5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:29.933980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc5b9-fac4-4ddd-9bb7-1ffc33417f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:29.935092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc5b9-fac4-4ddd-9bb7-1ffc33417f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:29.935982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc5b9-fac4-4ddd-9bb7-1ffc33417f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:29.936594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc5b9-fac4-4ddd-9bb7-1ffc33417f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:29.938152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc5b9-fac4-4ddd-9bb7-1ffc33417f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:29.940304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc5b9-fac4-4ddd-9bb7-1ffc33417f32 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:29.963607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01afe1-ed93-47a1-8901-2a17a69334e3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:29.964401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 997/2010] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:15:29.970268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3d01afe1-ed93-47a1-8901-2a17a69334e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:29.977035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc5b9-fac4-4ddd-9bb7-1ffc33417f32 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:29.996502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01afe1-ed93-47a1-8901-2a17a69334e3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:30.003125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc5b9-fac4-4ddd-9bb7-1ffc33417f32 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:30.008442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:30.009624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:30.010066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:30.010702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] PATH_INFO: `/v3/users/b375915231a14fc299367015ecfbf967` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:30.012295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b375915231a14fc299367015ecfbf967)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:30.012885 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc36cb-3f00-4684-8bdc-183d8248d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:30.014513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc36cb-3f00-4684-8bdc-183d8248d219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:30.015509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc36cb-3f00-4684-8bdc-183d8248d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:30.015849 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc36cb-3f00-4684-8bdc-183d8248d219 admin admin] PATH_INFO: `/v3/projects/d448706e6bff4b93b746096574062eb6/users/8b8e9d78546d4fd7a3b3090c4aff31c3/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:30.019759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:30.045674 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc36cb-3f00-4684-8bdc-183d8248d219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d448706e6bff4b93b746096574062eb6, user_id=8b8e9d78546d4fd7a3b3090c4aff31c3, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:30.050496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc36cb-3f00-4684-8bdc-183d8248d219 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:30.074698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 998/2011] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:29 2022] PUT /identity/v3/projects/d448706e6bff4b93b746096574062eb6/users/8b8e9d78546d4fd7a3b3090c4aff31c3/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:30.079984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc36cb-3f00-4684-8bdc-183d8248d219 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:30.149494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b375915231a14fc299367015ecfbf967'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:30.150387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b375915231a14fc299367015ecfbf967'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:30.160848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b375915231a14fc299367015ecfbf967'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:30.193273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc36cb-3f00-4684-8bdc-183d8248d219 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:30.196012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b375915231a14fc299367015ecfbf967'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:30.200035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c49f77-832e-4330-8be0-b94ca5bb7e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:30.201273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c49f77-832e-4330-8be0-b94ca5bb7e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:30.201750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c49f77-832e-4330-8be0-b94ca5bb7e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:30.202312 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c49f77-832e-4330-8be0-b94ca5bb7e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:30.203498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c49f77-832e-4330-8be0-b94ca5bb7e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:30.205991 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c49f77-832e-4330-8be0-b94ca5bb7e2f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:30.217299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f532dc0c-8241-4d1d-ad24-fdf81668a9f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b375915231a14fc299367015ecfbf967'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:30.226317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 999/2012] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:30.227338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c49f77-832e-4330-8be0-b94ca5bb7e2f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:30.229168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b5c49f77-832e-4330-8be0-b94ca5bb7e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:30.230365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1014/2013] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:29 2022] DELETE /identity/v3/users/b375915231a14fc299367015ecfbf967 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:30.237223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2435295-2b8f-499f-a574-d2b71e2cac37 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:30.238134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2435295-2b8f-499f-a574-d2b71e2cac37 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:30.242557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2435295-2b8f-499f-a574-d2b71e2cac37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:30.267008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c49f77-832e-4330-8be0-b94ca5bb7e2f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:30.276724 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e2435295-2b8f-499f-a574-d2b71e2cac37 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:30.278597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e2435295-2b8f-499f-a574-d2b71e2cac37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:30.361195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:30.396732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1015/2014] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:30.411701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2435295-2b8f-499f-a574-d2b71e2cac37 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:30.413513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e2435295-2b8f-499f-a574-d2b71e2cac37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:30.417260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20e7ee-eef3-497e-9868-55d57ae158d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:30.418592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20e7ee-eef3-497e-9868-55d57ae158d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:30.419321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20e7ee-eef3-497e-9868-55d57ae158d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:30.419831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20e7ee-eef3-497e-9868-55d57ae158d0 admin admin] PATH_INFO: `/v3/projects/d448706e6bff4b93b746096574062eb6/users/8b8e9d78546d4fd7a3b3090c4aff31c3/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:30.423805 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20e7ee-eef3-497e-9868-55d57ae158d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d448706e6bff4b93b746096574062eb6, user_id=8b8e9d78546d4fd7a3b3090c4aff31c3, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:30.426795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20e7ee-eef3-497e-9868-55d57ae158d0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:30.441364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1000/2015] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:30 2022] PUT /identity/v3/projects/d448706e6bff4b93b746096574062eb6/users/8b8e9d78546d4fd7a3b3090c4aff31c3/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:30.571922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2435295-2b8f-499f-a574-d2b71e2cac37 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:30.668646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d8874-21ab-4c36-aaa3-4fbed4dfaae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:30.670064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d8874-21ab-4c36-aaa3-4fbed4dfaae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:30.670752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d8874-21ab-4c36-aaa3-4fbed4dfaae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:30.672174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d8874-21ab-4c36-aaa3-4fbed4dfaae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:30.876212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d8874-21ab-4c36-aaa3-4fbed4dfaae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:30.877728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d8874-21ab-4c36-aaa3-4fbed4dfaae5 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:30.910966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1016/2016] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:32.173365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20e7ee-eef3-497e-9868-55d57ae158d0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:32.175876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ff20e7ee-eef3-497e-9868-55d57ae158d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:32.198091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20e7ee-eef3-497e-9868-55d57ae158d0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:32.223198 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:32.229267 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:32.229565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:32.229852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] PATH_INFO: `/v3/projects/bc1a73b47e7a41789b045738110dee21` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:32.236731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1a73b47e7a41789b045738110dee21)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:32.238944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:32.397443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:32.429394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1a73b47e7a41789b045738110dee21', 'request_id': 'req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:32.456652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1a73b47e7a41789b045738110dee21', 'request_id': 'req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:32.490961 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1a73b47e7a41789b045738110dee21', 'request_id': 'req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:32.530089 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] The token cache is being invalidate because project bc1a73b47e7a41789b045738110dee21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:32.531349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:32.532330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd016d9e-3a54-4da6-8c5c-b1715a8dd805 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:32.540490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1001/2017] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:32 2022] DELETE /identity/v3/projects/bc1a73b47e7a41789b045738110dee21 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:32.546613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74ee61-7019-49c0-9c91-362a1e94f5cd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:32.547346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74ee61-7019-49c0-9c91-362a1e94f5cd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:32.551966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74ee61-7019-49c0-9c91-362a1e94f5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:32.632837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9c74ee61-7019-49c0-9c91-362a1e94f5cd None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:32.635746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c74ee61-7019-49c0-9c91-362a1e94f5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:32.807306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1017/2018] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:32.811151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef47d-1021-427a-a183-1abead162485 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:32.811873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef47d-1021-427a-a183-1abead162485 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:32.813475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef47d-1021-427a-a183-1abead162485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:32.844190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-857ef47d-1021-427a-a183-1abead162485 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:32.846190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-857ef47d-1021-427a-a183-1abead162485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:32.865694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1002/2019] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:32.872353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74ee61-7019-49c0-9c91-362a1e94f5cd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:32.886292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:32.891596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74ee61-7019-49c0-9c91-362a1e94f5cd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:32.898962 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0095f3-8ac2-43e2-97fa-a4ec56ead76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:32.899837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0095f3-8ac2-43e2-97fa-a4ec56ead76c admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:32.900235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0095f3-8ac2-43e2-97fa-a4ec56ead76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:32.900626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0095f3-8ac2-43e2-97fa-a4ec56ead76c admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:32.902972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0095f3-8ac2-43e2-97fa-a4ec56ead76c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:32.903849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0095f3-8ac2-43e2-97fa-a4ec56ead76c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:32.907292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0095f3-8ac2-43e2-97fa-a4ec56ead76c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:32.934008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1018/2020] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:15:32.939862 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ef47d-1021-427a-a183-1abead162485 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:32.955488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ef47d-1021-427a-a183-1abead162485 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:32.964978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e7fde-5663-4c24-b597-c2dc1b27a122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:32.966126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e7fde-5663-4c24-b597-c2dc1b27a122 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:32.966573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e7fde-5663-4c24-b597-c2dc1b27a122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:32.967140 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e7fde-5663-4c24-b597-c2dc1b27a122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:32.969760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e7fde-5663-4c24-b597-c2dc1b27a122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:32.972175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e7fde-5663-4c24-b597-c2dc1b27a122 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:33.066695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1003/2021] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:32 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:15:33.071520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0095f3-8ac2-43e2-97fa-a4ec56ead76c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:33.104343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0095f3-8ac2-43e2-97fa-a4ec56ead76c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:33.220701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21296495-8ab9-4ef5-aea7-b9031b50fca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:33.221715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21296495-8ab9-4ef5-aea7-b9031b50fca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:33.222140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21296495-8ab9-4ef5-aea7-b9031b50fca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:33.222722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21296495-8ab9-4ef5-aea7-b9031b50fca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:33.225062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21296495-8ab9-4ef5-aea7-b9031b50fca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:33.228408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21296495-8ab9-4ef5-aea7-b9031b50fca7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:33.265987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1019/2022] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:33 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:33.268462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e7fde-5663-4c24-b597-c2dc1b27a122 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:33.316944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e7fde-5663-4c24-b597-c2dc1b27a122 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:33.348505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1df1df-2731-47f1-a512-261bcd0d2dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:33.349737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1df1df-2731-47f1-a512-261bcd0d2dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:33.350314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1df1df-2731-47f1-a512-261bcd0d2dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:33.350807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1df1df-2731-47f1-a512-261bcd0d2dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:33.354441 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1df1df-2731-47f1-a512-261bcd0d2dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:33.362344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1df1df-2731-47f1-a512-261bcd0d2dc5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:33.404252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21296495-8ab9-4ef5-aea7-b9031b50fca7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:33.413907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1004/2023] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:15:33.449328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21296495-8ab9-4ef5-aea7-b9031b50fca7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:33.457873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb20cd9-3ba0-49fc-b563-834dc97cbe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:33.458937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb20cd9-3ba0-49fc-b563-834dc97cbe54 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:33.460098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb20cd9-3ba0-49fc-b563-834dc97cbe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:33.460098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb20cd9-3ba0-49fc-b563-834dc97cbe54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:33.460514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb20cd9-3ba0-49fc-b563-834dc97cbe54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:33.478916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb20cd9-3ba0-49fc-b563-834dc97cbe54 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:33.491361 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1020/2024] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:33.503333 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1df1df-2731-47f1-a512-261bcd0d2dc5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:33.524535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1df1df-2731-47f1-a512-261bcd0d2dc5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:33.530549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ee96e-b891-4262-9253-c242d55470a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:33.531788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ee96e-b891-4262-9253-c242d55470a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:33.532444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ee96e-b891-4262-9253-c242d55470a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:33.532910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ee96e-b891-4262-9253-c242d55470a1 admin admin] PATH_INFO: `/v3/projects/9459130cbfe9402ab15590d79bd7edbb/users/27a03f08c37b4d0d8c5ea6cd87a98afe/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:33.553930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ee96e-b891-4262-9253-c242d55470a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9459130cbfe9402ab15590d79bd7edbb, user_id=27a03f08c37b4d0d8c5ea6cd87a98afe, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:33.557040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ee96e-b891-4262-9253-c242d55470a1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:33.603585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1005/2025] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:33 2022] PUT /identity/v3/projects/9459130cbfe9402ab15590d79bd7edbb/users/27a03f08c37b4d0d8c5ea6cd87a98afe/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:33.620196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e04f9-ef9e-4b08-ad2b-55080ea5a82b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:33.620972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e04f9-ef9e-4b08-ad2b-55080ea5a82b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:33.620972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e04f9-ef9e-4b08-ad2b-55080ea5a82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:33.685835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e59e04f9-ef9e-4b08-ad2b-55080ea5a82b None None] MFA Rules not processed for user `27a03f08c37b4d0d8c5ea6cd87a98afe`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:33.688956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e59e04f9-ef9e-4b08-ad2b-55080ea5a82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:33.889438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1021/2026] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:33.889532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ee96e-b891-4262-9253-c242d55470a1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:34.056262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ee96e-b891-4262-9253-c242d55470a1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:34.063760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aed3b-b29c-49b8-ad5e-2ca4b16a3689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:34.064469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aed3b-b29c-49b8-ad5e-2ca4b16a3689 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:34.064777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aed3b-b29c-49b8-ad5e-2ca4b16a3689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:34.065260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aed3b-b29c-49b8-ad5e-2ca4b16a3689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:34.066841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aed3b-b29c-49b8-ad5e-2ca4b16a3689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:34.069139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aed3b-b29c-49b8-ad5e-2ca4b16a3689 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:34.130646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1006/2027] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:33 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:34.132141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e04f9-ef9e-4b08-ad2b-55080ea5a82b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:34.159569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e04f9-ef9e-4b08-ad2b-55080ea5a82b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:34.276118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:34.289503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df697f4-27d0-4751-8c66-e430a730f113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:34.290217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df697f4-27d0-4751-8c66-e430a730f113 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:34.290510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df697f4-27d0-4751-8c66-e430a730f113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:34.290793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df697f4-27d0-4751-8c66-e430a730f113 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:34.291711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df697f4-27d0-4751-8c66-e430a730f113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:34.294023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df697f4-27d0-4751-8c66-e430a730f113 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:34.342695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1022/2028] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:34 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:34.347716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aed3b-b29c-49b8-ad5e-2ca4b16a3689 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:34.386720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aed3b-b29c-49b8-ad5e-2ca4b16a3689 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:34.397761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdedff-3e13-4d85-9185-d4b6ef19b4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:34.398601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdedff-3e13-4d85-9185-d4b6ef19b4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:34.399065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdedff-3e13-4d85-9185-d4b6ef19b4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:34.399552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdedff-3e13-4d85-9185-d4b6ef19b4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:34.400673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdedff-3e13-4d85-9185-d4b6ef19b4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:34.404516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdedff-3e13-4d85-9185-d4b6ef19b4c5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:34.434869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1007/2029] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:34.440436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df697f4-27d0-4751-8c66-e430a730f113 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:34.480485 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df697f4-27d0-4751-8c66-e430a730f113 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:34.496100 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78947e2e-337c-40c4-b791-f3354c3f3797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:34.496792 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78947e2e-337c-40c4-b791-f3354c3f3797 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:34.497141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78947e2e-337c-40c4-b791-f3354c3f3797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:34.497384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78947e2e-337c-40c4-b791-f3354c3f3797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:34.498526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78947e2e-337c-40c4-b791-f3354c3f3797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:34.500235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78947e2e-337c-40c4-b791-f3354c3f3797 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:34.518049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1023/2030] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:34.520283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdedff-3e13-4d85-9185-d4b6ef19b4c5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:34.533001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3362c-5e5b-453c-aedd-5d5e923f79bb None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:34.535642 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3362c-5e5b-453c-aedd-5d5e923f79bb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:34.536427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3362c-5e5b-453c-aedd-5d5e923f79bb None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:34.539027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1024/2031] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:15:34 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:15:34.554123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdedff-3e13-4d85-9185-d4b6ef19b4c5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:34.572330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b525b26-2b6a-44a2-8ff8-ca7f354de3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:34.574102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b525b26-2b6a-44a2-8ff8-ca7f354de3f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:34.574929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b525b26-2b6a-44a2-8ff8-ca7f354de3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:34.579094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae14a1-c832-49b7-9dc1-9bec0f7d3e4d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:34.579094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae14a1-c832-49b7-9dc1-9bec0f7d3e4d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:34.579094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae14a1-c832-49b7-9dc1-9bec0f7d3e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:34.581708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b525b26-2b6a-44a2-8ff8-ca7f354de3f2 admin admin] PATH_INFO: `/v3/projects/2de02fdf06c747369205dd8e59b2dc27/users/96e834f872c04018ba83c54ca46f8c37/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:34.616317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-37ae14a1-c832-49b7-9dc1-9bec0f7d3e4d None None] MFA Rules not processed for user `88ecd4c645174d28930a44b93cfee9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:34.617594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-37ae14a1-c832-49b7-9dc1-9bec0f7d3e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:34.659727 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b525b26-2b6a-44a2-8ff8-ca7f354de3f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de02fdf06c747369205dd8e59b2dc27, user_id=96e834f872c04018ba83c54ca46f8c37, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:34.667372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b525b26-2b6a-44a2-8ff8-ca7f354de3f2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:34.691585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1008/2032] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:34 2022] PUT /identity/v3/projects/2de02fdf06c747369205dd8e59b2dc27/users/96e834f872c04018ba83c54ca46f8c37/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:34.705043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec7182-5700-4133-897c-5395edcf3403 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:34.705391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec7182-5700-4133-897c-5395edcf3403 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:34.705680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec7182-5700-4133-897c-5395edcf3403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:34.768760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:34.775081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1025/2033] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:15:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:34.777357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae14a1-c832-49b7-9dc1-9bec0f7d3e4d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:34.781306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-37ae14a1-c832-49b7-9dc1-9bec0f7d3e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:34.783184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a7ec7182-5700-4133-897c-5395edcf3403 None None] MFA Rules not processed for user `96e834f872c04018ba83c54ca46f8c37`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:34.787847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a7ec7182-5700-4133-897c-5395edcf3403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:34.805401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae14a1-c832-49b7-9dc1-9bec0f7d3e4d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:34.950968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1009/2034] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:15:34.953025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec7182-5700-4133-897c-5395edcf3403 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:34.958966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a7ec7182-5700-4133-897c-5395edcf3403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:34.959160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:34.960339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:34.960699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:34.961051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] PATH_INFO: `/v3/users/5202e2bdba064c4aaa84e15e1f0f0dea` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:34.963162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5202e2bdba064c4aaa84e15e1f0f0dea)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:34.965341 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:35.067119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5202e2bdba064c4aaa84e15e1f0f0dea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.067905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5202e2bdba064c4aaa84e15e1f0f0dea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.091678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5202e2bdba064c4aaa84e15e1f0f0dea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.119229 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5202e2bdba064c4aaa84e15e1f0f0dea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.128595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec7182-5700-4133-897c-5395edcf3403 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:35.128857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:35.142381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5202e2bdba064c4aaa84e15e1f0f0dea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.166162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1026/2035] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:34 2022] DELETE /identity/v3/users/5202e2bdba064c4aaa84e15e1f0f0dea => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:35.166521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c8a0f-62a4-44c9-a53e-3affe707e9ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:35.166521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c8a0f-62a4-44c9-a53e-3affe707e9ba service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:35.166521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c8a0f-62a4-44c9-a53e-3affe707e9ba service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:35.166903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c8a0f-62a4-44c9-a53e-3affe707e9ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:35.168301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:35.172346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:35.176777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b00c8a0f-62a4-44c9-a53e-3affe707e9ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:35.217830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c8a0f-62a4-44c9-a53e-3affe707e9ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:35.223159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c8a0f-62a4-44c9-a53e-3affe707e9ba service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:35.273765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1010/2036] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:35.315944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a257996-89c7-4a69-9085-06ca1b55e9ae admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:35.342882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609370e-d385-4145-a542-baa0215e11b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:35.344103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609370e-d385-4145-a542-baa0215e11b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:35.344377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609370e-d385-4145-a542-baa0215e11b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:35.344663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609370e-d385-4145-a542-baa0215e11b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:35.345673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609370e-d385-4145-a542-baa0215e11b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:35.347782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609370e-d385-4145-a542-baa0215e11b2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:35.414764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c8a0f-62a4-44c9-a53e-3affe707e9ba service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:35.423586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1027/2037] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:35 2022] POST /identity/v3/projects => generated 375 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:35.433163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c8a0f-62a4-44c9-a53e-3affe707e9ba service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:35.582026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609370e-d385-4145-a542-baa0215e11b2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:35.584384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2609370e-d385-4145-a542-baa0215e11b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:35.622084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b35ff5-c673-432e-b8af-5e89f0fa0502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:35.622858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b35ff5-c673-432e-b8af-5e89f0fa0502 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:35.623150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b35ff5-c673-432e-b8af-5e89f0fa0502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:35.627355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b35ff5-c673-432e-b8af-5e89f0fa0502 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:35.628105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b35ff5-c673-432e-b8af-5e89f0fa0502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:35.634257 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b35ff5-c673-432e-b8af-5e89f0fa0502 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:35.705742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1011/2038] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:35 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:15:35.711191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b35ff5-c673-432e-b8af-5e89f0fa0502 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:35.731532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b35ff5-c673-432e-b8af-5e89f0fa0502 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:35.738177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb7f7e-f9ea-4c8b-8d9c-f489c905f456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:35.739705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb7f7e-f9ea-4c8b-8d9c-f489c905f456 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:35.740166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb7f7e-f9ea-4c8b-8d9c-f489c905f456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:35.740450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb7f7e-f9ea-4c8b-8d9c-f489c905f456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:35.741206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb7f7e-f9ea-4c8b-8d9c-f489c905f456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:35.743098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb7f7e-f9ea-4c8b-8d9c-f489c905f456 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:35.744907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609370e-d385-4145-a542-baa0215e11b2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:35.753302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:35.754580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:35.755092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:35.755676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] PATH_INFO: `/v3/projects/08cd35ed100244fdab3228d1afe62400` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:35.757696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=08cd35ed100244fdab3228d1afe62400)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:35.759022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1012/2039] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:35.761625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:35.768280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb7f7e-f9ea-4c8b-8d9c-f489c905f456 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:35.799618 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb7f7e-f9ea-4c8b-8d9c-f489c905f456 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:35.809979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b22c9-f361-493b-a50c-790ff4572ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:35.813534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b22c9-f361-493b-a50c-790ff4572ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:35.813909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b22c9-f361-493b-a50c-790ff4572ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:35.814429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b22c9-f361-493b-a50c-790ff4572ec8 admin admin] PATH_INFO: `/v3/projects/81c42fe4fc554172a8565ebe223c6547/users/67fe4417ade64da0a84bbf654ca547c5/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:35.847044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:35.866174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b22c9-f361-493b-a50c-790ff4572ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c42fe4fc554172a8565ebe223c6547, user_id=67fe4417ade64da0a84bbf654ca547c5, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:35.870871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b22c9-f361-493b-a50c-790ff4572ec8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:35.879027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08cd35ed100244fdab3228d1afe62400', 'request_id': 'req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.886609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08cd35ed100244fdab3228d1afe62400', 'request_id': 'req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.894953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08cd35ed100244fdab3228d1afe62400', 'request_id': 'req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.911770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1013/2040] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:35 2022] PUT /identity/v3/projects/81c42fe4fc554172a8565ebe223c6547/users/67fe4417ade64da0a84bbf654ca547c5/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:35.915919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b22c9-f361-493b-a50c-790ff4572ec8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:35.931649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] The token cache is being invalidate because project 08cd35ed100244fdab3228d1afe62400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:35.932349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.933566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-06e2a665-a2c9-4fe0-99b2-76df3aa8ab9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:35.937132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1028/2041] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:35 2022] DELETE /identity/v3/projects/08cd35ed100244fdab3228d1afe62400 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:35.943462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a86944-3882-41bb-9728-c1d6c87bd54c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:35.946107 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a86944-3882-41bb-9728-c1d6c87bd54c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:35.946934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a86944-3882-41bb-9728-c1d6c87bd54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:35.993408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:36.013430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c1a86944-3882-41bb-9728-c1d6c87bd54c None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:36.017111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c1a86944-3882-41bb-9728-c1d6c87bd54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:36.056088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b22c9-f361-493b-a50c-790ff4572ec8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:36.072716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be5769-c48d-49f1-9fec-3167a6db5e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:36.074848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be5769-c48d-49f1-9fec-3167a6db5e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:36.075139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be5769-c48d-49f1-9fec-3167a6db5e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:36.075464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be5769-c48d-49f1-9fec-3167a6db5e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:36.078110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be5769-c48d-49f1-9fec-3167a6db5e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:36.084888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be5769-c48d-49f1-9fec-3167a6db5e5f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:36.109635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1014/2042] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:36.114907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be5769-c48d-49f1-9fec-3167a6db5e5f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:36.119242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c9be5769-c48d-49f1-9fec-3167a6db5e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:36.149976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be5769-c48d-49f1-9fec-3167a6db5e5f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:36.157163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108a632-118f-4d29-b1c0-4f5f65c5c4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:36.158215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108a632-118f-4d29-b1c0-4f5f65c5c4d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:36.158578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108a632-118f-4d29-b1c0-4f5f65c5c4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:36.159108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108a632-118f-4d29-b1c0-4f5f65c5c4d3 admin admin] PATH_INFO: `/v3/projects/81c42fe4fc554172a8565ebe223c6547/users/67fe4417ade64da0a84bbf654ca547c5/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:36.163525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108a632-118f-4d29-b1c0-4f5f65c5c4d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c42fe4fc554172a8565ebe223c6547, user_id=67fe4417ade64da0a84bbf654ca547c5, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:36.172123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108a632-118f-4d29-b1c0-4f5f65c5c4d3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:36.202217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1029/2043] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:36.204172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a86944-3882-41bb-9728-c1d6c87bd54c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:36.206970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c1a86944-3882-41bb-9728-c1d6c87bd54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:36.219415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1015/2044] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:36 2022] PUT /identity/v3/projects/81c42fe4fc554172a8565ebe223c6547/users/67fe4417ade64da0a84bbf654ca547c5/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:15:36.221400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e6415-a725-406e-b6b8-7ed337669d9d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:36.221777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e6415-a725-406e-b6b8-7ed337669d9d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:36.222468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e6415-a725-406e-b6b8-7ed337669d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:36.251989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a86944-3882-41bb-9728-c1d6c87bd54c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:36.260510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fe5e6415-a725-406e-b6b8-7ed337669d9d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:36.261889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fe5e6415-a725-406e-b6b8-7ed337669d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:36.384211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1016/2045] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:36.386323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aec28-6b83-4b83-9d7c-816de27e1f66 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:36.386866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aec28-6b83-4b83-9d7c-816de27e1f66 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:36.387248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aec28-6b83-4b83-9d7c-816de27e1f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:36.419305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:36.420581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:36.421002 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:36.421448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] PATH_INFO: `/v3/users/fe8c83a278d24dd3bcdc0a80379e0ce9` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:36.423864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe8c83a278d24dd3bcdc0a80379e0ce9)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:36.427946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-945aec28-6b83-4b83-9d7c-816de27e1f66 None None] MFA Rules not processed for user `67fe4417ade64da0a84bbf654ca547c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:36.428735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:36.429360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-945aec28-6b83-4b83-9d7c-816de27e1f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:36.533939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe8c83a278d24dd3bcdc0a80379e0ce9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:36.535152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe8c83a278d24dd3bcdc0a80379e0ce9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:36.549248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe8c83a278d24dd3bcdc0a80379e0ce9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:36.576730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe8c83a278d24dd3bcdc0a80379e0ce9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:36.587767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:36.596827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe8c83a278d24dd3bcdc0a80379e0ce9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:36.609186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1030/2046] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:36 2022] DELETE /identity/v3/users/fe8c83a278d24dd3bcdc0a80379e0ce9 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:36.611034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:36.613713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:36.624774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1017/2047] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:36.624774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7208a4a-592a-4484-b7db-c92157977147 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:36.624774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7208a4a-592a-4484-b7db-c92157977147 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:36.624774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7208a4a-592a-4484-b7db-c92157977147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:36.664361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a7208a4a-592a-4484-b7db-c92157977147 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:36.665686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a7208a4a-592a-4484-b7db-c92157977147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:36.692825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:36.788834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8edc9f-a295-45c1-b2e7-191a7c150a6f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:36.797962 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:36.799027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:36.799687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:36.800003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:36.802528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:36.802947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:36.805380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:36.829001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1031/2048] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:15:36.829001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:36.830807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:36.831828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1018/2049] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:36.833403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7208a4a-592a-4484-b7db-c92157977147 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:36.857129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7208a4a-592a-4484-b7db-c92157977147 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:36.865444 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd1f9d-9c32-4290-8e19-ddf21d219178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:36.866953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd1f9d-9c32-4290-8e19-ddf21d219178 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:36.867450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd1f9d-9c32-4290-8e19-ddf21d219178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:36.868036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd1f9d-9c32-4290-8e19-ddf21d219178 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:36.869656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd1f9d-9c32-4290-8e19-ddf21d219178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:36.872905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd1f9d-9c32-4290-8e19-ddf21d219178 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:36.915390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1019/2050] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:36 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:36.917500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd1f9d-9c32-4290-8e19-ddf21d219178 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:36.920864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-30dd1f9d-9c32-4290-8e19-ddf21d219178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:36.938388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361a918-9fdf-4d82-b61d-8fc3756dfb6b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:37.025770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd1f9d-9c32-4290-8e19-ddf21d219178 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:37.027330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c9ee4-c9bf-453d-afc3-9eb71938e79f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:37.028336 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c9ee4-c9bf-453d-afc3-9eb71938e79f service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:37.029414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c9ee4-c9bf-453d-afc3-9eb71938e79f service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:37.029706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c9ee4-c9bf-453d-afc3-9eb71938e79f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:37.032958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b25c9ee4-c9bf-453d-afc3-9eb71938e79f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:37.034375 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061fc81-0505-49db-99cd-fef3e2cc7488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:37.036611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061fc81-0505-49db-99cd-fef3e2cc7488 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:37.037446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061fc81-0505-49db-99cd-fef3e2cc7488 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:37.038892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061fc81-0505-49db-99cd-fef3e2cc7488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:37.115604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c9ee4-c9bf-453d-afc3-9eb71938e79f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:37.117457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c9ee4-c9bf-453d-afc3-9eb71938e79f service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:37.128253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1032/2051] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:37.129304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c9ee4-c9bf-453d-afc3-9eb71938e79f service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:37.182536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061fc81-0505-49db-99cd-fef3e2cc7488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:37.185510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061fc81-0505-49db-99cd-fef3e2cc7488 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:37.194066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:37.213172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1020/2052] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:37.307304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c9ee4-c9bf-453d-afc3-9eb71938e79f service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:37.320389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7791e9b-2a99-4ea2-a91e-1d880b6531ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:37.321634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7791e9b-2a99-4ea2-a91e-1d880b6531ea admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:37.322069 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7791e9b-2a99-4ea2-a91e-1d880b6531ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:37.323369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7791e9b-2a99-4ea2-a91e-1d880b6531ea admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:37.324701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7791e9b-2a99-4ea2-a91e-1d880b6531ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:37.335510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7791e9b-2a99-4ea2-a91e-1d880b6531ea admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:37.390614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1033/2053] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:37 2022] POST /identity/v3/users => generated 294 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:37.397139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061fc81-0505-49db-99cd-fef3e2cc7488 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:37.434474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061fc81-0505-49db-99cd-fef3e2cc7488 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:37.458361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4274f-4a7a-4f81-8a40-a8816bb8588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:37.459415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4274f-4a7a-4f81-8a40-a8816bb8588b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:37.459918 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4274f-4a7a-4f81-8a40-a8816bb8588b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:37.460377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4274f-4a7a-4f81-8a40-a8816bb8588b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:37.461618 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4274f-4a7a-4f81-8a40-a8816bb8588b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:37.472020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4274f-4a7a-4f81-8a40-a8816bb8588b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:37.508519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1021/2054] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:37.509141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7791e9b-2a99-4ea2-a91e-1d880b6531ea admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:37.528328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7791e9b-2a99-4ea2-a91e-1d880b6531ea admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:37.540360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b40fc-ff82-45a2-8420-cdab0517b028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:37.543462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b40fc-ff82-45a2-8420-cdab0517b028 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:37.548918 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b40fc-ff82-45a2-8420-cdab0517b028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:37.550160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b40fc-ff82-45a2-8420-cdab0517b028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:37.552218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b40fc-ff82-45a2-8420-cdab0517b028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:37.555572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b40fc-ff82-45a2-8420-cdab0517b028 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:37.613893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1034/2055] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:15:37.619609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4274f-4a7a-4f81-8a40-a8816bb8588b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:37.639804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4274f-4a7a-4f81-8a40-a8816bb8588b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:37.648249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731410-ff00-45a6-864e-dba7b72027b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:37.649665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731410-ff00-45a6-864e-dba7b72027b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:37.650017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731410-ff00-45a6-864e-dba7b72027b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:37.651018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731410-ff00-45a6-864e-dba7b72027b8 admin admin] PATH_INFO: `/v3/projects/b745569c01e04aa18f5494b0856bc3d1/users/99798aca08904bc787f5d0afee8d0ac7/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:37.668474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:37.681569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94731410-ff00-45a6-864e-dba7b72027b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b745569c01e04aa18f5494b0856bc3d1, user_id=99798aca08904bc787f5d0afee8d0ac7, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:37.693137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94731410-ff00-45a6-864e-dba7b72027b8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:37.731976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1022/2056] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:37 2022] PUT /identity/v3/projects/b745569c01e04aa18f5494b0856bc3d1/users/99798aca08904bc787f5d0afee8d0ac7/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:37.744362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96cb5c-c236-40c3-ab1d-5ef097d3c4b8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:37.745872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96cb5c-c236-40c3-ab1d-5ef097d3c4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:37.746583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96cb5c-c236-40c3-ab1d-5ef097d3c4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:37.822477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-be96cb5c-c236-40c3-ab1d-5ef097d3c4b8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:37.823885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-be96cb5c-c236-40c3-ab1d-5ef097d3c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:37.842720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731410-ff00-45a6-864e-dba7b72027b8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:37.846098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-94731410-ff00-45a6-864e-dba7b72027b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:38.027864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1035/2057] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:38.038679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731410-ff00-45a6-864e-dba7b72027b8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:38.044721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:38.047607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96cb5c-c236-40c3-ab1d-5ef097d3c4b8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:38.049116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-be96cb5c-c236-40c3-ab1d-5ef097d3c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:38.057976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:38.058188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:38.058374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] PATH_INFO: `/v3/projects/2cb7e1b8314d4b3e9eb1d851f3efc5ca` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:38.064407 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb7e1b8314d4b3e9eb1d851f3efc5ca)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:38.070113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:38.196110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb7e1b8314d4b3e9eb1d851f3efc5ca', 'request_id': 'req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.206057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb7e1b8314d4b3e9eb1d851f3efc5ca', 'request_id': 'req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.211298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96cb5c-c236-40c3-ab1d-5ef097d3c4b8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:38.216204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb7e1b8314d4b3e9eb1d851f3efc5ca', 'request_id': 'req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.260836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] The token cache is being invalidate because project 2cb7e1b8314d4b3e9eb1d851f3efc5ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:38.261144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.263185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.279310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1023/2058] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:37 2022] DELETE /identity/v3/projects/2cb7e1b8314d4b3e9eb1d851f3efc5ca => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:38.293442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:38.298009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:38.322975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b02e4e-8af4-4f3f-98c2-82dbc20aa9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:38.323881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b02e4e-8af4-4f3f-98c2-82dbc20aa9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:38.324860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b02e4e-8af4-4f3f-98c2-82dbc20aa9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:38.325447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b02e4e-8af4-4f3f-98c2-82dbc20aa9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:38.328755 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-98b02e4e-8af4-4f3f-98c2-82dbc20aa9c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:38.370030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:38.437570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1692a73-58a1-4bbe-8e33-d0ac9ce92cf7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:38.447978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b02e4e-8af4-4f3f-98c2-82dbc20aa9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:38.449605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b02e4e-8af4-4f3f-98c2-82dbc20aa9c9 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:38.450604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:38.452518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:38.453148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:38.453837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] PATH_INFO: `/v3/users/33c4502b2643485fa8bb562d4f901370` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:38.457352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c4502b2643485fa8bb562d4f901370)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:38.461452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:38.469266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1036/2059] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:38.502881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:38.546922 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c4502b2643485fa8bb562d4f901370'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.557350 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c4502b2643485fa8bb562d4f901370'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.574850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c4502b2643485fa8bb562d4f901370'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.629732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c4502b2643485fa8bb562d4f901370'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.630935 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c4502b2643485fa8bb562d4f901370'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:38.658567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1024/2060] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:38 2022] DELETE /identity/v3/users/33c4502b2643485fa8bb562d4f901370 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:39.266018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b02e4e-8af4-4f3f-98c2-82dbc20aa9c9 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:39.295632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b02e4e-8af4-4f3f-98c2-82dbc20aa9c9 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:39.539518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:39.547295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:39.549627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:39.550104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] PATH_INFO: `/v3/projects/4c315501c78148cf973f8f9b732b86ab` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:39.556789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c315501c78148cf973f8f9b732b86ab)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:39.561009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:39.596963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:39.598459 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:39.699046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab574-998b-47d7-8644-570ee92ec240 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:39.728366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c315501c78148cf973f8f9b732b86ab', 'request_id': 'req-12194ca3-7f97-4207-b1d5-e592d4d413ee'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:39.745491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c315501c78148cf973f8f9b732b86ab', 'request_id': 'req-12194ca3-7f97-4207-b1d5-e592d4d413ee'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:39.758077 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c315501c78148cf973f8f9b732b86ab', 'request_id': 'req-12194ca3-7f97-4207-b1d5-e592d4d413ee'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:39.782305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] The token cache is being invalidate because project 4c315501c78148cf973f8f9b732b86ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:39.782759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:39.783525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-12194ca3-7f97-4207-b1d5-e592d4d413ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:39.790927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1037/2061] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:39 2022] DELETE /identity/v3/projects/4c315501c78148cf973f8f9b732b86ab => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:39.793749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f59389-430a-4ce8-8a69-b08747054cab None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:39.794125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f59389-430a-4ce8-8a69-b08747054cab None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:39.794678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f59389-430a-4ce8-8a69-b08747054cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:39.839888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ee0b8-38bf-4087-8dfd-517b2b234f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:39.841233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ee0b8-38bf-4087-8dfd-517b2b234f49 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:39.841973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ee0b8-38bf-4087-8dfd-517b2b234f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:39.842646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ee0b8-38bf-4087-8dfd-517b2b234f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:39.845868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e71ee0b8-38bf-4087-8dfd-517b2b234f49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:39.851135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-80f59389-430a-4ce8-8a69-b08747054cab None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:39.854217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-80f59389-430a-4ce8-8a69-b08747054cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:39.962537 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:40.010359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1038/2062] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:40.013751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a6723-4191-4852-bc1f-9895bddef0f7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:40.015051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a6723-4191-4852-bc1f-9895bddef0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:40.015641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a6723-4191-4852-bc1f-9895bddef0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:40.029033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ee0b8-38bf-4087-8dfd-517b2b234f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:40.036486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ee0b8-38bf-4087-8dfd-517b2b234f49 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:40.065062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-093a6723-4191-4852-bc1f-9895bddef0f7 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:40.068796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-093a6723-4191-4852-bc1f-9895bddef0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:40.070186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1025/2063] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:40.100888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ee0b8-38bf-4087-8dfd-517b2b234f49 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:40.101756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1039/2064] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:40.122033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ee0b8-38bf-4087-8dfd-517b2b234f49 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:40.143654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b720f8-2940-4580-9384-df54892f1e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:40.146162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b720f8-2940-4580-9384-df54892f1e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:40.146920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b720f8-2940-4580-9384-df54892f1e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:40.147642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b720f8-2940-4580-9384-df54892f1e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:40.148656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b720f8-2940-4580-9384-df54892f1e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:40.149339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b720f8-2940-4580-9384-df54892f1e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:40.151870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b720f8-2940-4580-9384-df54892f1e99 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:40.164077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:40.212668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a6723-4191-4852-bc1f-9895bddef0f7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:40.224096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1026/2065] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:15:40.237323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a6723-4191-4852-bc1f-9895bddef0f7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:40.251818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4446d4-c3c1-4bd1-a694-ecda66138c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:40.252945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4446d4-c3c1-4bd1-a694-ecda66138c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:40.253388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4446d4-c3c1-4bd1-a694-ecda66138c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:40.253809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4446d4-c3c1-4bd1-a694-ecda66138c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:40.255329 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4446d4-c3c1-4bd1-a694-ecda66138c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:40.258765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4446d4-c3c1-4bd1-a694-ecda66138c33 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:40.377913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b720f8-2940-4580-9384-df54892f1e99 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:40.380543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1040/2066] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:40 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:40.403228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b720f8-2940-4580-9384-df54892f1e99 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:40.580236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25087062-5082-4c7c-9a56-0ce26e83e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:40.582749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25087062-5082-4c7c-9a56-0ce26e83e906 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:40.583097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25087062-5082-4c7c-9a56-0ce26e83e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:40.583429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25087062-5082-4c7c-9a56-0ce26e83e906 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:40.586987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25087062-5082-4c7c-9a56-0ce26e83e906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:40.605703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25087062-5082-4c7c-9a56-0ce26e83e906 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:40.675018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1027/2067] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:40 2022] POST /identity/v3/users => generated 291 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:40.678939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4446d4-c3c1-4bd1-a694-ecda66138c33 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:40.697514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4446d4-c3c1-4bd1-a694-ecda66138c33 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:40.715698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06a3a2-84dd-4b5c-9599-bc096f6c5759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:40.716284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06a3a2-84dd-4b5c-9599-bc096f6c5759 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:40.716719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06a3a2-84dd-4b5c-9599-bc096f6c5759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:40.717235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06a3a2-84dd-4b5c-9599-bc096f6c5759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:40.719111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06a3a2-84dd-4b5c-9599-bc096f6c5759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:40.736655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06a3a2-84dd-4b5c-9599-bc096f6c5759 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:40.761179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1041/2068] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:40.768761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25087062-5082-4c7c-9a56-0ce26e83e906 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:40.792950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25087062-5082-4c7c-9a56-0ce26e83e906 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:40.802274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0ba7d-ab8b-42ea-b2cc-32eacfa5e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:40.806381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0ba7d-ab8b-42ea-b2cc-32eacfa5e8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:40.807449 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0ba7d-ab8b-42ea-b2cc-32eacfa5e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:40.808203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0ba7d-ab8b-42ea-b2cc-32eacfa5e8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:40.810705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0ba7d-ab8b-42ea-b2cc-32eacfa5e8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:40.814162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0ba7d-ab8b-42ea-b2cc-32eacfa5e8d3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:40.840271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1028/2069] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:40.842211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06a3a2-84dd-4b5c-9599-bc096f6c5759 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:40.876912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06a3a2-84dd-4b5c-9599-bc096f6c5759 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:40.888656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa7986-cf76-4db3-857b-6456119fad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:40.892197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa7986-cf76-4db3-857b-6456119fad3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:40.892911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa7986-cf76-4db3-857b-6456119fad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:40.893528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa7986-cf76-4db3-857b-6456119fad3b admin admin] PATH_INFO: `/v3/projects/0bcbf9aca2b74b208e60ef4325d9e430/users/81928e3b015a4caba5d7bcdba6b09000/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:40.933290 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa7986-cf76-4db3-857b-6456119fad3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcbf9aca2b74b208e60ef4325d9e430, user_id=81928e3b015a4caba5d7bcdba6b09000, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:40.938771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa7986-cf76-4db3-857b-6456119fad3b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:40.968289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1042/2070] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:40 2022] PUT /identity/v3/projects/0bcbf9aca2b74b208e60ef4325d9e430/users/81928e3b015a4caba5d7bcdba6b09000/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:40.989941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864db1c5-bd90-408e-bbcc-fbf907314151 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:40.990682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864db1c5-bd90-408e-bbcc-fbf907314151 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:40.991173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864db1c5-bd90-408e-bbcc-fbf907314151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:41.053306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-864db1c5-bd90-408e-bbcc-fbf907314151 None None] MFA Rules not processed for user `81928e3b015a4caba5d7bcdba6b09000`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:41.055130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-864db1c5-bd90-408e-bbcc-fbf907314151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:41.211185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1029/2071] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:41.216390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa7986-cf76-4db3-857b-6456119fad3b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:41.273391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d976a3-5dea-495d-8807-d92fdc00a24a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:41.274945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d976a3-5dea-495d-8807-d92fdc00a24a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:41.278775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d976a3-5dea-495d-8807-d92fdc00a24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:41.387115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-77d976a3-5dea-495d-8807-d92fdc00a24a None None] MFA Rules not processed for user `8b8e9d78546d4fd7a3b3090c4aff31c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:41.391658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-77d976a3-5dea-495d-8807-d92fdc00a24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:41.425076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa7986-cf76-4db3-857b-6456119fad3b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:41.446321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d11282-d1f9-43f7-b412-b8cbfc71515f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:41.447172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d11282-d1f9-43f7-b412-b8cbfc71515f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:41.447545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d11282-d1f9-43f7-b412-b8cbfc71515f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:41.447800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d11282-d1f9-43f7-b412-b8cbfc71515f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:41.448488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d11282-d1f9-43f7-b412-b8cbfc71515f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:41.450682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d11282-d1f9-43f7-b412-b8cbfc71515f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:41.476567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:41.512023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1030/2072] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:41.522813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d976a3-5dea-495d-8807-d92fdc00a24a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:41.524435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-77d976a3-5dea-495d-8807-d92fdc00a24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:41.543163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1043/2073] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:41 2022] POST /identity/v3/projects => generated 381 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:41.544837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d11282-d1f9-43f7-b412-b8cbfc71515f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:41.657298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d976a3-5dea-495d-8807-d92fdc00a24a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:41.672260 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d11282-d1f9-43f7-b412-b8cbfc71515f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:41.701379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2204ffa-e0ea-4f5f-9cfe-19523c0c49d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:41.710761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2204ffa-e0ea-4f5f-9cfe-19523c0c49d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:41.710761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2204ffa-e0ea-4f5f-9cfe-19523c0c49d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:41.710761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2204ffa-e0ea-4f5f-9cfe-19523c0c49d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:41.710761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2204ffa-e0ea-4f5f-9cfe-19523c0c49d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:41.715951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2204ffa-e0ea-4f5f-9cfe-19523c0c49d5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:41.747717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:41.774475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1044/2074] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:41 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:41.776120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2204ffa-e0ea-4f5f-9cfe-19523c0c49d5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:41.779966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f2204ffa-e0ea-4f5f-9cfe-19523c0c49d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:41.806742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bd381-d739-4e8f-bf49-5c24b123cb61 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92fc25f9ad7c49c48a909d2d92ea0a66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:41.807794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bd381-d739-4e8f-bf49-5c24b123cb61 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:41.808291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bd381-d739-4e8f-bf49-5c24b123cb61 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:41.808779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bd381-d739-4e8f-bf49-5c24b123cb61 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:41.812927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-468bd381-d739-4e8f-bf49-5c24b123cb61 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:41.887750 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2204ffa-e0ea-4f5f-9cfe-19523c0c49d5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:41.899320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106b413-df8f-49f9-8199-72f29d32cb69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:41.900146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b413-df8f-49f9-8199-72f29d32cb69 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:41.900540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b413-df8f-49f9-8199-72f29d32cb69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:41.901034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b413-df8f-49f9-8199-72f29d32cb69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:41.903828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a106b413-df8f-49f9-8199-72f29d32cb69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:42.016713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468bd381-d739-4e8f-bf49-5c24b123cb61 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:42.021236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468bd381-d739-4e8f-bf49-5c24b123cb61 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:42.056047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1031/2075] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:15:42.057712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bd381-d739-4e8f-bf49-5c24b123cb61 service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:42.067865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106b413-df8f-49f9-8199-72f29d32cb69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:42.072039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106b413-df8f-49f9-8199-72f29d32cb69 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:42.095231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1045/2076] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:42.106275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bd381-d739-4e8f-bf49-5c24b123cb61 service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:42.120936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01acda-dcc9-4739-867d-c7e5b0f2f3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:42.122154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01acda-dcc9-4739-867d-c7e5b0f2f3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:42.122584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01acda-dcc9-4739-867d-c7e5b0f2f3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:42.123799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01acda-dcc9-4739-867d-c7e5b0f2f3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:42.125717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01acda-dcc9-4739-867d-c7e5b0f2f3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:42.134476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01acda-dcc9-4739-867d-c7e5b0f2f3e6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:42.173304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1032/2077] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:42.175652 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106b413-df8f-49f9-8199-72f29d32cb69 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:42.197999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106b413-df8f-49f9-8199-72f29d32cb69 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:42.225477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10849c79-e1f2-4881-ac93-8b0fcbe31947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:42.241278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10849c79-e1f2-4881-ac93-8b0fcbe31947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:42.252863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10849c79-e1f2-4881-ac93-8b0fcbe31947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:42.252863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10849c79-e1f2-4881-ac93-8b0fcbe31947 admin admin] PATH_INFO: `/v3/projects/6342d46d397842bea6d6d0990658fc88/users/0ec0978933a54822a12452276ada6abb/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:42.324899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10849c79-e1f2-4881-ac93-8b0fcbe31947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6342d46d397842bea6d6d0990658fc88, user_id=0ec0978933a54822a12452276ada6abb, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:42.335819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10849c79-e1f2-4881-ac93-8b0fcbe31947 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:42.376216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1046/2078] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:42 2022] PUT /identity/v3/projects/6342d46d397842bea6d6d0990658fc88/users/0ec0978933a54822a12452276ada6abb/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:42.381621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01acda-dcc9-4739-867d-c7e5b0f2f3e6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:42.412985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01acda-dcc9-4739-867d-c7e5b0f2f3e6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:42.454417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aba201-9ede-4f5e-8471-fec23e990fff None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:42.455065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aba201-9ede-4f5e-8471-fec23e990fff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:42.455523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aba201-9ede-4f5e-8471-fec23e990fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:42.492674 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-36aba201-9ede-4f5e-8471-fec23e990fff None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:42.494838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-36aba201-9ede-4f5e-8471-fec23e990fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:42.536086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:42.570551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17bd9c-5405-4a87-9e02-64e09010cd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:42.572036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17bd9c-5405-4a87-9e02-64e09010cd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:42.572658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17bd9c-5405-4a87-9e02-64e09010cd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:42.573122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17bd9c-5405-4a87-9e02-64e09010cd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:42.575132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17bd9c-5405-4a87-9e02-64e09010cd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:42.578244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17bd9c-5405-4a87-9e02-64e09010cd92 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:42.621834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1033/2079] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:15:42.623088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17bd9c-5405-4a87-9e02-64e09010cd92 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:42.636120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17bd9c-5405-4a87-9e02-64e09010cd92 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:42.644810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c165f0d-cdb1-4895-b83c-2afa8f281bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:42.646302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c165f0d-cdb1-4895-b83c-2afa8f281bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:42.652735 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c165f0d-cdb1-4895-b83c-2afa8f281bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:42.654546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c165f0d-cdb1-4895-b83c-2afa8f281bcf admin admin] PATH_INFO: `/v3/projects/6342d46d397842bea6d6d0990658fc88/users/0ec0978933a54822a12452276ada6abb/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:42.665147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1047/2080] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:15:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:42.681957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c165f0d-cdb1-4895-b83c-2afa8f281bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6342d46d397842bea6d6d0990658fc88, user_id=0ec0978933a54822a12452276ada6abb, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:42.689174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c165f0d-cdb1-4895-b83c-2afa8f281bcf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:42.736513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1034/2081] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:42 2022] PUT /identity/v3/projects/6342d46d397842bea6d6d0990658fc88/users/0ec0978933a54822a12452276ada6abb/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:42.747581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313c973-36a9-418c-a440-487910bc68ff None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:42.747957 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313c973-36a9-418c-a440-487910bc68ff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:42.748243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313c973-36a9-418c-a440-487910bc68ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:42.752981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c165f0d-cdb1-4895-b83c-2afa8f281bcf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:42.760249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c165f0d-cdb1-4895-b83c-2afa8f281bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:42.804826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9313c973-36a9-418c-a440-487910bc68ff None None] MFA Rules not processed for user `0ec0978933a54822a12452276ada6abb`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:42.806830 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9313c973-36a9-418c-a440-487910bc68ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:42.852900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:42.892391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c165f0d-cdb1-4895-b83c-2afa8f281bcf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:42.899636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bcd9f-1441-42a2-b9ec-d647d9d6062b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:42.900526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bcd9f-1441-42a2-b9ec-d647d9d6062b service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:42.901033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bcd9f-1441-42a2-b9ec-d647d9d6062b service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:42.901493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bcd9f-1441-42a2-b9ec-d647d9d6062b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:42.964186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1048/2082] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:42.985853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313c973-36a9-418c-a440-487910bc68ff None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:43.013617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bcd9f-1441-42a2-b9ec-d647d9d6062b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:43.017471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bcd9f-1441-42a2-b9ec-d647d9d6062b service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:43.035641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1035/2083] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:15:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:15:43.130032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313c973-36a9-418c-a440-487910bc68ff None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:43.136813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761142c-dac4-4337-961e-cf536b05fabe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:43.139338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761142c-dac4-4337-961e-cf536b05fabe service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:43.139861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761142c-dac4-4337-961e-cf536b05fabe service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:43.140328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761142c-dac4-4337-961e-cf536b05fabe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:43.172818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761142c-dac4-4337-961e-cf536b05fabe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:43.175388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761142c-dac4-4337-961e-cf536b05fabe service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:43.206404 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1049/2084] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:44.363412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bcd9f-1441-42a2-b9ec-d647d9d6062b service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:44.382970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bcd9f-1441-42a2-b9ec-d647d9d6062b service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:44.408220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e6d72-285d-4cfa-86ab-bc02947d4488 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ec0978933a54822a12452276ada6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6342d46d397842bea6d6d0990658fc88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:44.409229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e6d72-285d-4cfa-86ab-bc02947d4488 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:44.411037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e6d72-285d-4cfa-86ab-bc02947d4488 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:44.412870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e6d72-285d-4cfa-86ab-bc02947d4488 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:44.414197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e6d72-285d-4cfa-86ab-bc02947d4488 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:44.414544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e6d72-285d-4cfa-86ab-bc02947d4488 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:44.421462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e6d72-285d-4cfa-86ab-bc02947d4488 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:44.461078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1036/2085] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:15:44.465903 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761142c-dac4-4337-961e-cf536b05fabe service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:44.476918 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761142c-dac4-4337-961e-cf536b05fabe service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:44.508651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7b391-9450-4906-9227-3788cfeb8c8b tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ec0978933a54822a12452276ada6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6342d46d397842bea6d6d0990658fc88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:44.509668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7b391-9450-4906-9227-3788cfeb8c8b tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:44.509668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7b391-9450-4906-9227-3788cfeb8c8b tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:44.510270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7b391-9450-4906-9227-3788cfeb8c8b tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:44.510695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7b391-9450-4906-9227-3788cfeb8c8b tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:44.535160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7b391-9450-4906-9227-3788cfeb8c8b tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:44.638859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1050/2086] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:44 2022] POST /identity/v3/projects => generated 355 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:44.640638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e6d72-285d-4cfa-86ab-bc02947d4488 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:44.667563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e6d72-285d-4cfa-86ab-bc02947d4488 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:44.967080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ec0978933a54822a12452276ada6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6342d46d397842bea6d6d0990658fc88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:44.968529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:44.969022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:44.969499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:44.970824 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:44.971379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:44.974139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:44.986648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:45.030248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1037/2087] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:15:45.102396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7b391-9450-4906-9227-3788cfeb8c8b tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:45.128441 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7b391-9450-4906-9227-3788cfeb8c8b tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:45.140333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ec0978933a54822a12452276ada6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6342d46d397842bea6d6d0990658fc88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:45.142234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:45.142548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:45.142854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] PATH_INFO: `/v3/projects/740f15e6b9754ecf9bfd2b4e65d1e488` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:45.145705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Authorizing `identity:delete_project(project_id=740f15e6b9754ecf9bfd2b4e65d1e488)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:45.149363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:45.372137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '740f15e6b9754ecf9bfd2b4e65d1e488', 'request_id': 'req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:45.386188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '740f15e6b9754ecf9bfd2b4e65d1e488', 'request_id': 'req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:45.401434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '740f15e6b9754ecf9bfd2b4e65d1e488', 'request_id': 'req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:45.434374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] The token cache is being invalidate because project 740f15e6b9754ecf9bfd2b4e65d1e488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:45.439790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:45.443296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ad6a194-da9e-49d1-a165-b61e6e4b8e0e tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:45.452992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1051/2088] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:45 2022] DELETE /identity/v3/projects/740f15e6b9754ecf9bfd2b4e65d1e488 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:46.139151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:46.145318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:46.318870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95dc0-5fa7-48d2-aaae-dd3471abfa53 tempest-VolumeQuotaClassesTest-1451651607 tempest-VolumeQuotaClassesTest-1451651607-project-admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:46.492955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:46.494650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:46.494650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:46.494650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] PATH_INFO: `/v3/users/81928e3b015a4caba5d7bcdba6b09000` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:46.497958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=81928e3b015a4caba5d7bcdba6b09000)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:46.502870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:46.643491 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81928e3b015a4caba5d7bcdba6b09000'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:46.660927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81928e3b015a4caba5d7bcdba6b09000'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:46.689260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81928e3b015a4caba5d7bcdba6b09000'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:46.721944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81928e3b015a4caba5d7bcdba6b09000'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:46.730888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81928e3b015a4caba5d7bcdba6b09000'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:46.744769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1038/2089] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:46 2022] DELETE /identity/v3/users/81928e3b015a4caba5d7bcdba6b09000 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:46.749842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb39073-72f9-4e91-bc42-628078a94466 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:46.750599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb39073-72f9-4e91-bc42-628078a94466 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:46.750599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb39073-72f9-4e91-bc42-628078a94466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:46.791519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1fb39073-72f9-4e91-bc42-628078a94466 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:46.792909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1fb39073-72f9-4e91-bc42-628078a94466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:46.827982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:46.966491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1052/2090] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:46.973225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:46.975117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:47.028997 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:47.146743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa0818-7837-4bf4-9ead-d24ca34324ee admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:47.157931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be39e3-f35e-4da2-aa00-9d05616a0fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:47.158761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be39e3-f35e-4da2-aa00-9d05616a0fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:47.159060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be39e3-f35e-4da2-aa00-9d05616a0fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:47.159367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be39e3-f35e-4da2-aa00-9d05616a0fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:47.185207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be39e3-f35e-4da2-aa00-9d05616a0fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:47.187279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be39e3-f35e-4da2-aa00-9d05616a0fd9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:47.239937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1039/2091] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:47.852363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb39073-72f9-4e91-bc42-628078a94466 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:47.854752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1fb39073-72f9-4e91-bc42-628078a94466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:47.877727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb39073-72f9-4e91-bc42-628078a94466 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:47.885548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:47.887161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:47.887823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:47.888386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] PATH_INFO: `/v3/users/27a03f08c37b4d0d8c5ea6cd87a98afe` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:47.890954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a03f08c37b4d0d8c5ea6cd87a98afe)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:47.894366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:48.003435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a03f08c37b4d0d8c5ea6cd87a98afe'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:48.004591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a03f08c37b4d0d8c5ea6cd87a98afe'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:48.018750 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a03f08c37b4d0d8c5ea6cd87a98afe'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:48.048355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a03f08c37b4d0d8c5ea6cd87a98afe'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:48.089778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a03f08c37b4d0d8c5ea6cd87a98afe'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:48.121953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1053/2092] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:47 2022] DELETE /identity/v3/users/27a03f08c37b4d0d8c5ea6cd87a98afe => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:48.128604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422159f-46d8-42e7-bea4-6fa8b7a0232f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:48.129185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422159f-46d8-42e7-bea4-6fa8b7a0232f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:48.129770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422159f-46d8-42e7-bea4-6fa8b7a0232f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:48.166934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1422159f-46d8-42e7-bea4-6fa8b7a0232f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:48.169083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1422159f-46d8-42e7-bea4-6fa8b7a0232f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:48.320863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1040/2093] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:48.417817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:48.420414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:48.638172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6a195-083c-4c49-8f2b-5fd8a75103cb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:48.648097 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee3181-0de0-4aa6-a4ad-809ba20286bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:48.667492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee3181-0de0-4aa6-a4ad-809ba20286bf service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:48.668144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee3181-0de0-4aa6-a4ad-809ba20286bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:48.668936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee3181-0de0-4aa6-a4ad-809ba20286bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:48.681926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422159f-46d8-42e7-bea4-6fa8b7a0232f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:48.683766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1422159f-46d8-42e7-bea4-6fa8b7a0232f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:48.722438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422159f-46d8-42e7-bea4-6fa8b7a0232f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:48.756298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee3181-0de0-4aa6-a4ad-809ba20286bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:48.757514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee3181-0de0-4aa6-a4ad-809ba20286bf service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:48.764943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:48.765868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:48.766151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:48.766431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] PATH_INFO: `/v3/projects/0bcbf9aca2b74b208e60ef4325d9e430` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:48.769921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bcbf9aca2b74b208e60ef4325d9e430)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:48.773779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:48.791876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1054/2094] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:48.793524 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4defa-40a7-40f1-9f7c-3ee8e7252c53 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:48.793829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4defa-40a7-40f1-9f7c-3ee8e7252c53 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:48.794117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4defa-40a7-40f1-9f7c-3ee8e7252c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:48.893472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e2c4defa-40a7-40f1-9f7c-3ee8e7252c53 None None] MFA Rules not processed for user `99798aca08904bc787f5d0afee8d0ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:48.896682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e2c4defa-40a7-40f1-9f7c-3ee8e7252c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:48.938664 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bcbf9aca2b74b208e60ef4325d9e430', 'request_id': 'req-410d5fac-76f9-416a-b522-7ce6732bbc49'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:48.964986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bcbf9aca2b74b208e60ef4325d9e430', 'request_id': 'req-410d5fac-76f9-416a-b522-7ce6732bbc49'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:48.974413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:48.994481 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bcbf9aca2b74b208e60ef4325d9e430', 'request_id': 'req-410d5fac-76f9-416a-b522-7ce6732bbc49'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.042508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] The token cache is being invalidate because project 0bcbf9aca2b74b208e60ef4325d9e430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:49.042809 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.047979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.055679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1041/2095] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:48 2022] DELETE /identity/v3/projects/0bcbf9aca2b74b208e60ef4325d9e430 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:49.060667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:49.066631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:49.126288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1055/2096] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:49.163981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4defa-40a7-40f1-9f7c-3ee8e7252c53 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:49.165457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e2c4defa-40a7-40f1-9f7c-3ee8e7252c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:49.286957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:49.368650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4defa-40a7-40f1-9f7c-3ee8e7252c53 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:49.389022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219a9d9-bbe6-43cd-b2df-fd07130498fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:49.390725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219a9d9-bbe6-43cd-b2df-fd07130498fd service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:49.390725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219a9d9-bbe6-43cd-b2df-fd07130498fd service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:49.391059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219a9d9-bbe6-43cd-b2df-fd07130498fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:49.402476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d5fac-76f9-416a-b522-7ce6732bbc49 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:49.410731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219a9d9-bbe6-43cd-b2df-fd07130498fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:49.412806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219a9d9-bbe6-43cd-b2df-fd07130498fd service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:49.423402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:49.425073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:49.425650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:49.430763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] PATH_INFO: `/v3/users/0ec0978933a54822a12452276ada6abb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:49.437089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1056/2097] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:49.437978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219a9d9-bbe6-43cd-b2df-fd07130498fd service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:49.438484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec0978933a54822a12452276ada6abb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:49.439403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c219a9d9-bbe6-43cd-b2df-fd07130498fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:49.442146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:49.470910 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219a9d9-bbe6-43cd-b2df-fd07130498fd service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:49.484885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:49.485821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:49.486115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:49.486430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] PATH_INFO: `/v3/projects/9459130cbfe9402ab15590d79bd7edbb` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:49.492479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9459130cbfe9402ab15590d79bd7edbb)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:49.495235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:49.628847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec0978933a54822a12452276ada6abb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.641787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec0978933a54822a12452276ada6abb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.663098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec0978933a54822a12452276ada6abb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.670229 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9459130cbfe9402ab15590d79bd7edbb', 'request_id': 'req-16aa5c98-d0ed-4871-9591-a5cf3e835304'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.681665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9459130cbfe9402ab15590d79bd7edbb', 'request_id': 'req-16aa5c98-d0ed-4871-9591-a5cf3e835304'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.696680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9459130cbfe9402ab15590d79bd7edbb', 'request_id': 'req-16aa5c98-d0ed-4871-9591-a5cf3e835304'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.703025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec0978933a54822a12452276ada6abb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.707316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec0978933a54822a12452276ada6abb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.751153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1042/2098] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:49 2022] DELETE /identity/v3/users/0ec0978933a54822a12452276ada6abb => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:49.765921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] The token cache is being invalidate because project 9459130cbfe9402ab15590d79bd7edbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:49.767347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.771852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:49.776791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1057/2099] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:49 2022] DELETE /identity/v3/projects/9459130cbfe9402ab15590d79bd7edbb => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:49.782588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:49.786338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:49.976543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3a7f7-1458-443d-841b-4df2ec236907 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:49.987353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:49.988954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:49.991050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:49.991754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] PATH_INFO: `/v3/users/96e834f872c04018ba83c54ca46f8c37` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:49.994753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e834f872c04018ba83c54ca46f8c37)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:49.997740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:50.164224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e834f872c04018ba83c54ca46f8c37'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:50.197445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e834f872c04018ba83c54ca46f8c37'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:50.231384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e834f872c04018ba83c54ca46f8c37'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:50.243141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:50.277273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e834f872c04018ba83c54ca46f8c37'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:50.282671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b501ea24-3da1-401f-a409-df3358bb472a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e834f872c04018ba83c54ca46f8c37'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:50.315670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1043/2100] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:49 2022] DELETE /identity/v3/users/96e834f872c04018ba83c54ca46f8c37 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:50.451669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:50.455275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:50.576725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:50.728689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa5c98-d0ed-4871-9591-a5cf3e835304 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:51.002949 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:51.004548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:51.005045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:51.005517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] PATH_INFO: `/v3/projects/6342d46d397842bea6d6d0990658fc88` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:51.010789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6342d46d397842bea6d6d0990658fc88)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:51.013089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:51.154638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6342d46d397842bea6d6d0990658fc88', 'request_id': 'req-4790a541-aeba-4467-ad1c-e03a9c61e713'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.178109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6342d46d397842bea6d6d0990658fc88', 'request_id': 'req-4790a541-aeba-4467-ad1c-e03a9c61e713'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.190689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6342d46d397842bea6d6d0990658fc88', 'request_id': 'req-4790a541-aeba-4467-ad1c-e03a9c61e713'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.252054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] The token cache is being invalidate because project 6342d46d397842bea6d6d0990658fc88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:51.252311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.253804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.262878 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1058/2101] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:50 2022] DELETE /identity/v3/projects/6342d46d397842bea6d6d0990658fc88 => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:51.265552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707545b-6237-49fe-86c3-9fc7ce73151e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:51.266067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707545b-6237-49fe-86c3-9fc7ce73151e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:51.266565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707545b-6237-49fe-86c3-9fc7ce73151e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:51.329654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:51.336851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:51.369425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5707545b-6237-49fe-86c3-9fc7ce73151e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:51.378898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5707545b-6237-49fe-86c3-9fc7ce73151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:51.551353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1044/2102] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:15:51.558761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579f636-44e7-453c-8161-41fc30d1032b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:51.559092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579f636-44e7-453c-8161-41fc30d1032b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:51.559433 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579f636-44e7-453c-8161-41fc30d1032b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:51.568922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790a541-aeba-4467-ad1c-e03a9c61e713 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:51.582299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:51.584356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:51.584961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:51.585465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] PATH_INFO: `/v3/projects/2de02fdf06c747369205dd8e59b2dc27` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:51.587377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de02fdf06c747369205dd8e59b2dc27)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:51.590143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:51.612882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5579f636-44e7-453c-8161-41fc30d1032b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:51.615209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5579f636-44e7-453c-8161-41fc30d1032b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:51.621318 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:51.650772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1045/2103] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:51.654015 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579f636-44e7-453c-8161-41fc30d1032b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:51.676105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579f636-44e7-453c-8161-41fc30d1032b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:51.711365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de02fdf06c747369205dd8e59b2dc27', 'request_id': 'req-2eb9835e-b924-4065-a69c-6b6f97e15106'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.730614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de02fdf06c747369205dd8e59b2dc27', 'request_id': 'req-2eb9835e-b924-4065-a69c-6b6f97e15106'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.760832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de02fdf06c747369205dd8e59b2dc27', 'request_id': 'req-2eb9835e-b924-4065-a69c-6b6f97e15106'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.850404 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] The token cache is being invalidate because project 2de02fdf06c747369205dd8e59b2dc27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:51.850945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.852334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:51.861946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1059/2104] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:51 2022] DELETE /identity/v3/projects/2de02fdf06c747369205dd8e59b2dc27 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:51.862967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:51.864831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:51.916770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:51.917764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:51.918448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:51.918448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:51.920443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:51.920443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:51.922479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:52.005962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1046/2105] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:15:52.015193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:52.025738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:52.056278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225d3fc-10f0-4e68-b6cc-c8fa2df96366 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:52.074179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77a074-497c-456a-bb46-3cfaac832485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:52.076158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77a074-497c-456a-bb46-3cfaac832485 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:52.077950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77a074-497c-456a-bb46-3cfaac832485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:52.079520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9835e-b924-4065-a69c-6b6f97e15106 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:52.084662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77a074-497c-456a-bb46-3cfaac832485 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:52.088046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77a074-497c-456a-bb46-3cfaac832485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:52.088522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:52.091312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:52.091312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:52.091312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] PATH_INFO: `/v3/users/67fe4417ade64da0a84bbf654ca547c5` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:52.095338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67fe4417ade64da0a84bbf654ca547c5)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:52.099411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:52.099891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77a074-497c-456a-bb46-3cfaac832485 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:52.192996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1047/2106] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:52 2022] POST /identity/v3/projects => generated 393 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:52.195032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77a074-497c-456a-bb46-3cfaac832485 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:52.265351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67fe4417ade64da0a84bbf654ca547c5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:52.266615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67fe4417ade64da0a84bbf654ca547c5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:52.281239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67fe4417ade64da0a84bbf654ca547c5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:52.332333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67fe4417ade64da0a84bbf654ca547c5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:52.342785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67fe4417ade64da0a84bbf654ca547c5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:52.351738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1060/2107] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:15:51 2022] DELETE /identity/v3/users/67fe4417ade64da0a84bbf654ca547c5 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:52.369149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:52.397457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77a074-497c-456a-bb46-3cfaac832485 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:52.627575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c0fb-a7d7-40ad-940b-654b4464c9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:52.628389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c0fb-a7d7-40ad-940b-654b4464c9dd admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:52.628861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c0fb-a7d7-40ad-940b-654b4464c9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:52.629233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c0fb-a7d7-40ad-940b-654b4464c9dd admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:52.630784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9c0fb-a7d7-40ad-940b-654b4464c9dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:52.643779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9c0fb-a7d7-40ad-940b-654b4464c9dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:52.686136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1048/2108] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:52 2022] POST /identity/v3/users => generated 297 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:52.688369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:52.690223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:52.736259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32ccf6-dfcb-4005-b932-28b7fc7ad546 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:52.749748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce357e93-1f64-435a-9e72-f558ed2f36e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:52.750444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce357e93-1f64-435a-9e72-f558ed2f36e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:52.750733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce357e93-1f64-435a-9e72-f558ed2f36e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:52.751372 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce357e93-1f64-435a-9e72-f558ed2f36e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:52.751985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce357e93-1f64-435a-9e72-f558ed2f36e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:52.757734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce357e93-1f64-435a-9e72-f558ed2f36e1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:52.799754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1061/2109] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:52.805166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c0fb-a7d7-40ad-940b-654b4464c9dd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:52.867053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c0fb-a7d7-40ad-940b-654b4464c9dd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:52.895736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2719fb-6d3f-4db6-b4bd-f2684433da7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:52.901154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2719fb-6d3f-4db6-b4bd-f2684433da7e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:52.904426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2719fb-6d3f-4db6-b4bd-f2684433da7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:52.906137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2719fb-6d3f-4db6-b4bd-f2684433da7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:52.915919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2719fb-6d3f-4db6-b4bd-f2684433da7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:52.932896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2719fb-6d3f-4db6-b4bd-f2684433da7e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:52.996483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce357e93-1f64-435a-9e72-f558ed2f36e1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:53.010793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1049/2110] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:53.022927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce357e93-1f64-435a-9e72-f558ed2f36e1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:53.042982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136e491-5ee6-4601-b083-894bb853789b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:53.044747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136e491-5ee6-4601-b083-894bb853789b admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:53.045157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136e491-5ee6-4601-b083-894bb853789b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:53.046482 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136e491-5ee6-4601-b083-894bb853789b admin admin] PATH_INFO: `/v3/projects/134c9c750fba4c409746df2592780cb0/users/0adaf93cec97476ca688e4cf5a9eb0b7/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:53.050988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:53.116431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136e491-5ee6-4601-b083-894bb853789b admin admin] RBAC: Authorizing `identity:create_grant(project_id=134c9c750fba4c409746df2592780cb0, user_id=0adaf93cec97476ca688e4cf5a9eb0b7, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:53.127904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136e491-5ee6-4601-b083-894bb853789b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:53.193952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1062/2111] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:52 2022] PUT /identity/v3/projects/134c9c750fba4c409746df2592780cb0/users/0adaf93cec97476ca688e4cf5a9eb0b7/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:15:53.209260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c976be3-31bc-4110-8311-73ec75acc531 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:53.212274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c976be3-31bc-4110-8311-73ec75acc531 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:53.212665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c976be3-31bc-4110-8311-73ec75acc531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:53.275425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3c976be3-31bc-4110-8311-73ec75acc531 None None] MFA Rules not processed for user `0adaf93cec97476ca688e4cf5a9eb0b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:53.275425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3c976be3-31bc-4110-8311-73ec75acc531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:53.429861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1050/2112] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:53.434693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136e491-5ee6-4601-b083-894bb853789b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:53.556160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c976be3-31bc-4110-8311-73ec75acc531 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:53.558722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3c976be3-31bc-4110-8311-73ec75acc531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:53.613952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136e491-5ee6-4601-b083-894bb853789b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:53.641708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68905267-ed1a-4fe9-8157-902f83edf06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:53.643355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68905267-ed1a-4fe9-8157-902f83edf06e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:53.643964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68905267-ed1a-4fe9-8157-902f83edf06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:53.660823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68905267-ed1a-4fe9-8157-902f83edf06e admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:53.661940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68905267-ed1a-4fe9-8157-902f83edf06e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:53.665262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68905267-ed1a-4fe9-8157-902f83edf06e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:53.700665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c976be3-31bc-4110-8311-73ec75acc531 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:53.717613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:53.719136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:53.719919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:53.720346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] PATH_INFO: `/v3/projects/81c42fe4fc554172a8565ebe223c6547` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:53.727033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81c42fe4fc554172a8565ebe223c6547)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:53.729203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:53.757493 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1063/2113] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:53 2022] POST /identity/v3/projects => generated 391 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:53.759216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68905267-ed1a-4fe9-8157-902f83edf06e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:53.890361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81c42fe4fc554172a8565ebe223c6547', 'request_id': 'req-808ec536-b0d5-40d8-a22a-7b2ea817eb58'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:53.909282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81c42fe4fc554172a8565ebe223c6547', 'request_id': 'req-808ec536-b0d5-40d8-a22a-7b2ea817eb58'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:53.921269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:53.925249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81c42fe4fc554172a8565ebe223c6547', 'request_id': 'req-808ec536-b0d5-40d8-a22a-7b2ea817eb58'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:53.965689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] The token cache is being invalidate because project 81c42fe4fc554172a8565ebe223c6547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:15:53.966161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:53.967937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-808ec536-b0d5-40d8-a22a-7b2ea817eb58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:15:53.970965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1051/2114] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:15:53 2022] DELETE /identity/v3/projects/81c42fe4fc554172a8565ebe223c6547 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:53.973512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2bd6f-e7c4-4de6-8f0f-084e6ad26a1d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:53.973512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2bd6f-e7c4-4de6-8f0f-084e6ad26a1d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:53.973512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2bd6f-e7c4-4de6-8f0f-084e6ad26a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.004557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68905267-ed1a-4fe9-8157-902f83edf06e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.014639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b5d2bd6f-e7c4-4de6-8f0f-084e6ad26a1d None None] MFA Rules not processed for user `99798aca08904bc787f5d0afee8d0ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:54.017888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b5d2bd6f-e7c4-4de6-8f0f-084e6ad26a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:54.134440 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba63bb-f056-41e2-b652-2d72b8025687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:54.135243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba63bb-f056-41e2-b652-2d72b8025687 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.135683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba63bb-f056-41e2-b652-2d72b8025687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.136251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba63bb-f056-41e2-b652-2d72b8025687 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.136874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba63bb-f056-41e2-b652-2d72b8025687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:54.139136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba63bb-f056-41e2-b652-2d72b8025687 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:54.145694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1052/2115] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:54.145694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14e2ef-999a-4200-85b8-76dd3eaaa3c0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.145694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14e2ef-999a-4200-85b8-76dd3eaaa3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.145694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14e2ef-999a-4200-85b8-76dd3eaaa3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.169728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8e14e2ef-999a-4200-85b8-76dd3eaaa3c0 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:54.172296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8e14e2ef-999a-4200-85b8-76dd3eaaa3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:54.183114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1053/2116] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:54.199176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fa873-d174-455c-ac7c-ab4661438736 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.199524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fa873-d174-455c-ac7c-ab4661438736 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.200260 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1064/2117] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:53 2022] POST /identity/v3/users => generated 302 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:54.201537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fa873-d174-455c-ac7c-ab4661438736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.201768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba63bb-f056-41e2-b652-2d72b8025687 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:54.204080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-48ba63bb-f056-41e2-b652-2d72b8025687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:54.225639 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:54.234230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba63bb-f056-41e2-b652-2d72b8025687 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.244216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8e9fa873-d174-455c-ac7c-ab4661438736 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:54.244503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd593085-039b-4afc-ab1c-53ea8f010c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:54.245415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd593085-039b-4afc-ab1c-53ea8f010c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.245546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8e9fa873-d174-455c-ac7c-ab4661438736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:54.246066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd593085-039b-4afc-ab1c-53ea8f010c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.246680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd593085-039b-4afc-ab1c-53ea8f010c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.247951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd593085-039b-4afc-ab1c-53ea8f010c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:54.251344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd593085-039b-4afc-ab1c-53ea8f010c90 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:54.260777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1054/2118] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:15:54.267429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fa873-d174-455c-ac7c-ab4661438736 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:54.285173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fa873-d174-455c-ac7c-ab4661438736 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.295364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1065/2119] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:54.299037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd593085-039b-4afc-ab1c-53ea8f010c90 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:54.304497 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afb883-8a9c-4dee-b271-7176fc68b226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:54.305762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afb883-8a9c-4dee-b271-7176fc68b226 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.306417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afb883-8a9c-4dee-b271-7176fc68b226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.307320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afb883-8a9c-4dee-b271-7176fc68b226 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.308730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afb883-8a9c-4dee-b271-7176fc68b226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:15:54.309463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afb883-8a9c-4dee-b271-7176fc68b226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:54.312987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afb883-8a9c-4dee-b271-7176fc68b226 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:54.319285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd593085-039b-4afc-ab1c-53ea8f010c90 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.335228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1055/2120] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:15:54.336894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aea890-4730-48f0-9005-5a5e6230a1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:54.337981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aea890-4730-48f0-9005-5a5e6230a1cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.338414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aea890-4730-48f0-9005-5a5e6230a1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.338987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aea890-4730-48f0-9005-5a5e6230a1cb admin admin] PATH_INFO: `/v3/projects/da94548f338f4da0abf2a4617bb287fc/users/1f16cb67b197478aa6d0ec14f37cda17/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.342237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afb883-8a9c-4dee-b271-7176fc68b226 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:54.372737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afb883-8a9c-4dee-b271-7176fc68b226 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.373137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aea890-4730-48f0-9005-5a5e6230a1cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=da94548f338f4da0abf2a4617bb287fc, user_id=1f16cb67b197478aa6d0ec14f37cda17, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:54.377653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aea890-4730-48f0-9005-5a5e6230a1cb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:54.389991 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feb2d9-0b75-4379-aefc-746158e4bab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:54.389991 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feb2d9-0b75-4379-aefc-746158e4bab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.390616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feb2d9-0b75-4379-aefc-746158e4bab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.391086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feb2d9-0b75-4379-aefc-746158e4bab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.391892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feb2d9-0b75-4379-aefc-746158e4bab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:54.397852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feb2d9-0b75-4379-aefc-746158e4bab7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:54.434571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1066/2121] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:54 2022] PUT /identity/v3/projects/da94548f338f4da0abf2a4617bb287fc/users/1f16cb67b197478aa6d0ec14f37cda17/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:54.451329 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aea890-4730-48f0-9005-5a5e6230a1cb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:54.491251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1056/2122] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:15:54 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:15:54.493306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feb2d9-0b75-4379-aefc-746158e4bab7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:54.599466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aea890-4730-48f0-9005-5a5e6230a1cb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.661840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feb2d9-0b75-4379-aefc-746158e4bab7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.694022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f395591-50b9-4eed-b56d-389c324e83eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:54.695462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f395591-50b9-4eed-b56d-389c324e83eb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.695768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f395591-50b9-4eed-b56d-389c324e83eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.696059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f395591-50b9-4eed-b56d-389c324e83eb admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.696830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f395591-50b9-4eed-b56d-389c324e83eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:54.699149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f395591-50b9-4eed-b56d-389c324e83eb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:54.762721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1057/2123] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:15:54 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:15:54.770921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f395591-50b9-4eed-b56d-389c324e83eb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:54.773075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af05b4-6725-4861-8746-b101aae613d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:54.773773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af05b4-6725-4861-8746-b101aae613d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.778355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af05b4-6725-4861-8746-b101aae613d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.779148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af05b4-6725-4861-8746-b101aae613d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.780836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af05b4-6725-4861-8746-b101aae613d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:54.786273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af05b4-6725-4861-8746-b101aae613d2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:54.803456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f395591-50b9-4eed-b56d-389c324e83eb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.822131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6b260-9d71-4115-a5ec-d0b42940480f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:54.823161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6b260-9d71-4115-a5ec-d0b42940480f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.823385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1067/2124] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:54.823634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6b260-9d71-4115-a5ec-d0b42940480f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.823978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6b260-9d71-4115-a5ec-d0b42940480f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.824972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6b260-9d71-4115-a5ec-d0b42940480f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:54.827875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af05b4-6725-4861-8746-b101aae613d2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:54.835158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6b260-9d71-4115-a5ec-d0b42940480f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:54.850935 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af05b4-6725-4861-8746-b101aae613d2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.860980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1058/2125] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:54.867854 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6b260-9d71-4115-a5ec-d0b42940480f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:54.869067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ade89e-4b25-4ed8-b6b7-5c2a3bd1a9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:54.870230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ade89e-4b25-4ed8-b6b7-5c2a3bd1a9da admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.870697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ade89e-4b25-4ed8-b6b7-5c2a3bd1a9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.871136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ade89e-4b25-4ed8-b6b7-5c2a3bd1a9da admin admin] PATH_INFO: `/v3/projects/da94548f338f4da0abf2a4617bb287fc/users/1f16cb67b197478aa6d0ec14f37cda17/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.875084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ade89e-4b25-4ed8-b6b7-5c2a3bd1a9da admin admin] RBAC: Authorizing `identity:create_grant(project_id=da94548f338f4da0abf2a4617bb287fc, user_id=1f16cb67b197478aa6d0ec14f37cda17, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:54.879388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ade89e-4b25-4ed8-b6b7-5c2a3bd1a9da admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:54.890556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6b260-9d71-4115-a5ec-d0b42940480f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:54.900577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1068/2126] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:54 2022] PUT /identity/v3/projects/da94548f338f4da0abf2a4617bb287fc/users/1f16cb67b197478aa6d0ec14f37cda17/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:54.914178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f5710-95f3-43c1-b476-c2d28284f9ae None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:54.914938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f5710-95f3-43c1-b476-c2d28284f9ae None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:54.916805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f5710-95f3-43c1-b476-c2d28284f9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:54.971071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c63f5710-95f3-43c1-b476-c2d28284f9ae None None] MFA Rules not processed for user `1f16cb67b197478aa6d0ec14f37cda17`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:54.972995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c63f5710-95f3-43c1-b476-c2d28284f9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:54.978366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:55.051599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe47d7-7056-475d-ba40-805d14050ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:55.052310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe47d7-7056-475d-ba40-805d14050ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:55.052724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe47d7-7056-475d-ba40-805d14050ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:55.053005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe47d7-7056-475d-ba40-805d14050ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:55.053754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe47d7-7056-475d-ba40-805d14050ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:55.055960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe47d7-7056-475d-ba40-805d14050ca1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:55.074016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1059/2127] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:15:55.081871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe47d7-7056-475d-ba40-805d14050ca1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:55.098406 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe47d7-7056-475d-ba40-805d14050ca1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:55.104838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68602e0-8a75-41d3-9b28-fa4051d930f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:55.105807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:55.105942 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68602e0-8a75-41d3-9b28-fa4051d930f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:55.106188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68602e0-8a75-41d3-9b28-fa4051d930f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:55.106465 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68602e0-8a75-41d3-9b28-fa4051d930f7 admin admin] PATH_INFO: `/v3/projects/7e769192a87447de9f788a2fba3b4009/users/5fceafd63ed84a019fed9ba232e90ac3/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:55.142917 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68602e0-8a75-41d3-9b28-fa4051d930f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e769192a87447de9f788a2fba3b4009, user_id=5fceafd63ed84a019fed9ba232e90ac3, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:55.148383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68602e0-8a75-41d3-9b28-fa4051d930f7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:55.179635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1060/2128] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:15:55 2022] PUT /identity/v3/projects/7e769192a87447de9f788a2fba3b4009/users/5fceafd63ed84a019fed9ba232e90ac3/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:15:55.193597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62f09f-1683-4b1c-8ac3-dea7aec566e2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:55.193836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62f09f-1683-4b1c-8ac3-dea7aec566e2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:55.194018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62f09f-1683-4b1c-8ac3-dea7aec566e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:55.229728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1069/2129] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:15:55.242589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f5710-95f3-43c1-b476-c2d28284f9ae None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:55.245016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c63f5710-95f3-43c1-b476-c2d28284f9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:55.287851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6c62f09f-1683-4b1c-8ac3-dea7aec566e2 None None] MFA Rules not processed for user `5fceafd63ed84a019fed9ba232e90ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:15:55.290557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6c62f09f-1683-4b1c-8ac3-dea7aec566e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:55.430238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1061/2130] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:15:55.431919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f5710-95f3-43c1-b476-c2d28284f9ae None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:55.440795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62f09f-1683-4b1c-8ac3-dea7aec566e2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:55.442597 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6c62f09f-1683-4b1c-8ac3-dea7aec566e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:55.559860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431097d-b6c2-475e-87e4-c66628a7e97a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:55.562084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431097d-b6c2-475e-87e4-c66628a7e97a service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:55.564299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62f09f-1683-4b1c-8ac3-dea7aec566e2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:55.565679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431097d-b6c2-475e-87e4-c66628a7e97a service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:55.566458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431097d-b6c2-475e-87e4-c66628a7e97a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:55.569987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0e502-3e3e-4865-a654-3ddd40ffc71a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92fc25f9ad7c49c48a909d2d92ea0a66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:55.570826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0e502-3e3e-4865-a654-3ddd40ffc71a service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:55.571425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0e502-3e3e-4865-a654-3ddd40ffc71a service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:55.571823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0e502-3e3e-4865-a654-3ddd40ffc71a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:55.584275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0e502-3e3e-4865-a654-3ddd40ffc71a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:55.585913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0e502-3e3e-4865-a654-3ddd40ffc71a service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:55.620883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1062/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:55.804993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431097d-b6c2-475e-87e4-c66628a7e97a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:55.807742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431097d-b6c2-475e-87e4-c66628a7e97a service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:55.846546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1070/2132] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:15:56.018400 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0e502-3e3e-4865-a654-3ddd40ffc71a service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:15:56.096808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0e502-3e3e-4865-a654-3ddd40ffc71a service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:15:56.125589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d30c8d-6c7c-4dd6-8fc9-a4046a9e3a35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:15:56.127101 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d30c8d-6c7c-4dd6-8fc9-a4046a9e3a35 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:15:56.127616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d30c8d-6c7c-4dd6-8fc9-a4046a9e3a35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:15:56.130707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d30c8d-6c7c-4dd6-8fc9-a4046a9e3a35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:15:56.136272 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c0d30c8d-6c7c-4dd6-8fc9-a4046a9e3a35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:15:56.194684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:15:56.259568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d30c8d-6c7c-4dd6-8fc9-a4046a9e3a35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:15:56.262799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d30c8d-6c7c-4dd6-8fc9-a4046a9e3a35 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:15:56.298551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1063/2133] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:15:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:16:01.996832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431097d-b6c2-475e-87e4-c66628a7e97a service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:02.104622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:02.149731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431097d-b6c2-475e-87e4-c66628a7e97a service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:02.161250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:02.162266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:02.162591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:02.162985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] PATH_INFO: `/v3/users/0adaf93cec97476ca688e4cf5a9eb0b7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:02.166576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0adaf93cec97476ca688e4cf5a9eb0b7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:02.169147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:02.297455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0adaf93cec97476ca688e4cf5a9eb0b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:02.298337 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0adaf93cec97476ca688e4cf5a9eb0b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:02.318173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0adaf93cec97476ca688e4cf5a9eb0b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:02.352520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0adaf93cec97476ca688e4cf5a9eb0b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:02.368516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0adaf93cec97476ca688e4cf5a9eb0b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:02.383056 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1071/2134] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:16:01 2022] DELETE /identity/v3/users/0adaf93cec97476ca688e4cf5a9eb0b7 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:02.385394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a21fa-596f-4149-b7a0-cfec6e0c7697 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:02.385999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a21fa-596f-4149-b7a0-cfec6e0c7697 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:02.386324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a21fa-596f-4149-b7a0-cfec6e0c7697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:02.456315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7b4a21fa-596f-4149-b7a0-cfec6e0c7697 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:02.474074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7b4a21fa-596f-4149-b7a0-cfec6e0c7697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:02.680779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1064/2135] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:16:02.708291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:02.710391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:02.815064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c570d-2222-4697-a38c-302f2c34d8f5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:02.819350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb7682-b6b6-42bd-87ce-e2fcfb8997fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:02.820135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb7682-b6b6-42bd-87ce-e2fcfb8997fd service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:02.820503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb7682-b6b6-42bd-87ce-e2fcfb8997fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:02.820858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb7682-b6b6-42bd-87ce-e2fcfb8997fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:02.834879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb7682-b6b6-42bd-87ce-e2fcfb8997fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:02.837359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb7682-b6b6-42bd-87ce-e2fcfb8997fd service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:02.863732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1072/2136] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:16:03.867251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a21fa-596f-4149-b7a0-cfec6e0c7697 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:03.872484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7b4a21fa-596f-4149-b7a0-cfec6e0c7697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:03.899982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a21fa-596f-4149-b7a0-cfec6e0c7697 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:03.926235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:03.932261 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:03.932846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:03.934863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] PATH_INFO: `/v3/projects/134c9c750fba4c409746df2592780cb0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:03.938173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=134c9c750fba4c409746df2592780cb0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:03.949867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:04.255318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '134c9c750fba4c409746df2592780cb0', 'request_id': 'req-62a50fbd-ac78-4934-b75f-a72b5e15a784'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.296598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '134c9c750fba4c409746df2592780cb0', 'request_id': 'req-62a50fbd-ac78-4934-b75f-a72b5e15a784'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.323827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '134c9c750fba4c409746df2592780cb0', 'request_id': 'req-62a50fbd-ac78-4934-b75f-a72b5e15a784'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.376975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] The token cache is being invalidate because project 134c9c750fba4c409746df2592780cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:16:04.377899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.382289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.388317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1065/2137] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:16:03 2022] DELETE /identity/v3/projects/134c9c750fba4c409746df2592780cb0 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:04.394155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb7682-b6b6-42bd-87ce-e2fcfb8997fd service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:04.409040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb7682-b6b6-42bd-87ce-e2fcfb8997fd service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:04.557479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:04.559082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:04.559646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:04.560139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] PATH_INFO: `/v3/users/1f16cb67b197478aa6d0ec14f37cda17` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:04.562287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f16cb67b197478aa6d0ec14f37cda17)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:04.565784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:04.607596 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:04.764629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f16cb67b197478aa6d0ec14f37cda17'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.769682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f16cb67b197478aa6d0ec14f37cda17'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.786779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f16cb67b197478aa6d0ec14f37cda17'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.831490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f16cb67b197478aa6d0ec14f37cda17'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.868871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ada41c01-6cde-43af-a6cd-742ee815682f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f16cb67b197478aa6d0ec14f37cda17'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:04.890462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1073/2138] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:16:04 2022] DELETE /identity/v3/users/1f16cb67b197478aa6d0ec14f37cda17 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:05.689276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:05.691627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:05.745590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a50fbd-ac78-4934-b75f-a72b5e15a784 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:05.866828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:05.888416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:05.889186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:05.889517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:05.889952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] PATH_INFO: `/v3/projects/da94548f338f4da0abf2a4617bb287fc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:05.891562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da94548f338f4da0abf2a4617bb287fc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:05.893536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:05.974908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da94548f338f4da0abf2a4617bb287fc', 'request_id': 'req-77b3a6da-4252-4b7f-8a25-8bb2002165b9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:05.983022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da94548f338f4da0abf2a4617bb287fc', 'request_id': 'req-77b3a6da-4252-4b7f-8a25-8bb2002165b9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:05.992525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da94548f338f4da0abf2a4617bb287fc', 'request_id': 'req-77b3a6da-4252-4b7f-8a25-8bb2002165b9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:06.025552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] The token cache is being invalidate because project da94548f338f4da0abf2a4617bb287fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:16:06.030150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:06.033232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77b3a6da-4252-4b7f-8a25-8bb2002165b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:06.049906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1066/2139] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:16:05 2022] DELETE /identity/v3/projects/da94548f338f4da0abf2a4617bb287fc => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:06.051896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7faf4-b9bb-47ee-b7cc-23082361f666 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.052399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7faf4-b9bb-47ee-b7cc-23082361f666 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.053158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7faf4-b9bb-47ee-b7cc-23082361f666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.105182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c1b7faf4-b9bb-47ee-b7cc-23082361f666 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:06.107253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c1b7faf4-b9bb-47ee-b7cc-23082361f666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:06.281420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1074/2140] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:06.294857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a674b0-3fc4-4b6b-a9ef-c3ea04e1cbb4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.295861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a674b0-3fc4-4b6b-a9ef-c3ea04e1cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.296410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a674b0-3fc4-4b6b-a9ef-c3ea04e1cbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.326142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-76a674b0-3fc4-4b6b-a9ef-c3ea04e1cbb4 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:06.330122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-76a674b0-3fc4-4b6b-a9ef-c3ea04e1cbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:06.348687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1067/2141] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:06.353763 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7faf4-b9bb-47ee-b7cc-23082361f666 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:06.370317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7faf4-b9bb-47ee-b7cc-23082361f666 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:06.379129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be246c2-55bf-4ee8-8386-2a08c584f0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:06.380306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be246c2-55bf-4ee8-8386-2a08c584f0ef admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.380821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be246c2-55bf-4ee8-8386-2a08c584f0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.381328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be246c2-55bf-4ee8-8386-2a08c584f0ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.382711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be246c2-55bf-4ee8-8386-2a08c584f0ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:16:06.383310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be246c2-55bf-4ee8-8386-2a08c584f0ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:06.386491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be246c2-55bf-4ee8-8386-2a08c584f0ef admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:06.407804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1075/2142] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:16:06.413304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a674b0-3fc4-4b6b-a9ef-c3ea04e1cbb4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:06.424564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a674b0-3fc4-4b6b-a9ef-c3ea04e1cbb4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:06.430172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9cc7c-77f7-46ec-aa0f-c30e64eec586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:06.430888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9cc7c-77f7-46ec-aa0f-c30e64eec586 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.431210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9cc7c-77f7-46ec-aa0f-c30e64eec586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.431529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9cc7c-77f7-46ec-aa0f-c30e64eec586 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.432202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9cc7c-77f7-46ec-aa0f-c30e64eec586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:06.434418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9cc7c-77f7-46ec-aa0f-c30e64eec586 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:06.472495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1068/2143] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:16:06 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:16:06.479413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be246c2-55bf-4ee8-8386-2a08c584f0ef admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:06.492363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be246c2-55bf-4ee8-8386-2a08c584f0ef admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:06.526926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11d219-94ee-489a-b705-00edd4380206 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.527249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11d219-94ee-489a-b705-00edd4380206 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.527566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11d219-94ee-489a-b705-00edd4380206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.556815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4b11d219-94ee-489a-b705-00edd4380206 None None] MFA Rules not processed for user `e908d2b09dfc4a139d9c27f52580e15b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:06.558093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b11d219-94ee-489a-b705-00edd4380206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:06.622445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e9734-da20-48ea-9f2b-7b8334beabec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:06.624536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e9734-da20-48ea-9f2b-7b8334beabec admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.624536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e9734-da20-48ea-9f2b-7b8334beabec admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.624536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e9734-da20-48ea-9f2b-7b8334beabec admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.625184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e9734-da20-48ea-9f2b-7b8334beabec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:06.627222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e9734-da20-48ea-9f2b-7b8334beabec admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:06.654550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1076/2144] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:16:06 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:16:06.660310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e9734-da20-48ea-9f2b-7b8334beabec admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:06.673792 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e9734-da20-48ea-9f2b-7b8334beabec admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:06.680031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01bb5b-960d-47af-8cda-265d65440ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:06.680782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01bb5b-960d-47af-8cda-265d65440ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.681118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01bb5b-960d-47af-8cda-265d65440ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.681411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01bb5b-960d-47af-8cda-265d65440ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.682218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01bb5b-960d-47af-8cda-265d65440ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:06.683902 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01bb5b-960d-47af-8cda-265d65440ac2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:06.687791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1069/2145] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:06.688427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:06.703907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1077/2146] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:16:06.708894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11d219-94ee-489a-b705-00edd4380206 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:06.731697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11d219-94ee-489a-b705-00edd4380206 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:06.739524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4ea73-f299-4bf6-a236-77d096ef994b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:06.741490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4ea73-f299-4bf6-a236-77d096ef994b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.741490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4ea73-f299-4bf6-a236-77d096ef994b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.741490 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4ea73-f299-4bf6-a236-77d096ef994b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.742512 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4ea73-f299-4bf6-a236-77d096ef994b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:06.744842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4ea73-f299-4bf6-a236-77d096ef994b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:06.768035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1070/2147] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:16:06.768280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01bb5b-960d-47af-8cda-265d65440ac2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:06.781320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01bb5b-960d-47af-8cda-265d65440ac2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:06.787966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5916c-9f1c-412e-b674-5e642f31705d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:06.789434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5916c-9f1c-412e-b674-5e642f31705d admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.789928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5916c-9f1c-412e-b674-5e642f31705d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.790374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5916c-9f1c-412e-b674-5e642f31705d admin admin] PATH_INFO: `/v3/projects/ecb9b72f8b5c487e8b8339f487197cb1/users/3097cdb231fd47e3a9e598c57803362a/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.821356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5916c-9f1c-412e-b674-5e642f31705d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb9b72f8b5c487e8b8339f487197cb1, user_id=3097cdb231fd47e3a9e598c57803362a, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:06.827717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5916c-9f1c-412e-b674-5e642f31705d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:06.862141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1078/2148] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:16:06 2022] PUT /identity/v3/projects/ecb9b72f8b5c487e8b8339f487197cb1/users/3097cdb231fd47e3a9e598c57803362a/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:06.875166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252a291-270e-458e-8ced-6c7c248a985a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:06.875528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252a291-270e-458e-8ced-6c7c248a985a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:06.875822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252a291-270e-458e-8ced-6c7c248a985a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:06.916779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4252a291-270e-458e-8ced-6c7c248a985a None None] MFA Rules not processed for user `3097cdb231fd47e3a9e598c57803362a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:06.918038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4252a291-270e-458e-8ced-6c7c248a985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:06.984710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:07.033866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1071/2149] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:16:07.053182 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5916c-9f1c-412e-b674-5e642f31705d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:07.055744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-60e5916c-9f1c-412e-b674-5e642f31705d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:07.172022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5916c-9f1c-412e-b674-5e642f31705d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:07.178336 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1baaf-a981-4e4f-9c84-35c685eb748a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:07.179523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1baaf-a981-4e4f-9c84-35c685eb748a service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:07.180005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1baaf-a981-4e4f-9c84-35c685eb748a service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:07.180489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1baaf-a981-4e4f-9c84-35c685eb748a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:07.196104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1baaf-a981-4e4f-9c84-35c685eb748a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:07.198986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1baaf-a981-4e4f-9c84-35c685eb748a service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:07.224396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1079/2150] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:16:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:16:09.465639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4b4d5-aff4-412b-bae8-c5857d205c95 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:09.466216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4b4d5-aff4-412b-bae8-c5857d205c95 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:09.466542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4b4d5-aff4-412b-bae8-c5857d205c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:09.495866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f8f4b4d5-aff4-412b-bae8-c5857d205c95 None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:09.497162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f8f4b4d5-aff4-412b-bae8-c5857d205c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:09.588698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1072/2151] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:16:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:09.616650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1baaf-a981-4e4f-9c84-35c685eb748a service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:09.618954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f8b1baaf-a981-4e4f-9c84-35c685eb748a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:09.729115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1baaf-a981-4e4f-9c84-35c685eb748a service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:09.736760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0b494-ec76-4396-aed6-0f7ef553a367 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:09.737760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0b494-ec76-4396-aed6-0f7ef553a367 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:09.738187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0b494-ec76-4396-aed6-0f7ef553a367 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:09.738563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0b494-ec76-4396-aed6-0f7ef553a367 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:09.751144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0b494-ec76-4396-aed6-0f7ef553a367 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:09.754073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0b494-ec76-4396-aed6-0f7ef553a367 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:09.770504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1080/2152] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:16:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:16:19.578894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=36 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:19.764146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:33.796383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a29a3-761b-49be-b80c-27c12c8389c2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:33.797179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a29a3-761b-49be-b80c-27c12c8389c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:33.798245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a29a3-761b-49be-b80c-27c12c8389c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:33.827845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-898a29a3-761b-49be-b80c-27c12c8389c2 None None] MFA Rules not processed for user `691877dce1024aeda61629073d74602b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:33.829587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-898a29a3-761b-49be-b80c-27c12c8389c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:33.904334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1073/2153] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:34.570396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0b494-ec76-4396-aed6-0f7ef553a367 service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:34.674378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0b494-ec76-4396-aed6-0f7ef553a367 service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:34.680530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:34.681530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:34.681898 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:34.682256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] PATH_INFO: `/v3/users/3097cdb231fd47e3a9e598c57803362a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:34.684030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=3097cdb231fd47e3a9e598c57803362a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:34.687101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:34.785506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3097cdb231fd47e3a9e598c57803362a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:34.788329 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3097cdb231fd47e3a9e598c57803362a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:34.799408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3097cdb231fd47e3a9e598c57803362a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:34.833934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3097cdb231fd47e3a9e598c57803362a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:34.850219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3097cdb231fd47e3a9e598c57803362a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:34.861386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1081/2154] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:16:34 2022] DELETE /identity/v3/users/3097cdb231fd47e3a9e598c57803362a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:34.868903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389ea1-cd44-4319-9d09-657bb071c0e2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:34.871091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389ea1-cd44-4319-9d09-657bb071c0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:34.872090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389ea1-cd44-4319-9d09-657bb071c0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:34.901068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ea389ea1-cd44-4319-9d09-657bb071c0e2 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:34.903922 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ea389ea1-cd44-4319-9d09-657bb071c0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:35.013094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1074/2155] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:35.024583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:35.026986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:35.197637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3297d1f-6fae-46a1-b1b1-5007cff003ce admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:35.202728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc547d00-c673-41ba-a549-2fc9512a754b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:35.206304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc547d00-c673-41ba-a549-2fc9512a754b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:35.207592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc547d00-c673-41ba-a549-2fc9512a754b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:35.209968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc547d00-c673-41ba-a549-2fc9512a754b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:35.234036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc547d00-c673-41ba-a549-2fc9512a754b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:35.237245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc547d00-c673-41ba-a549-2fc9512a754b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:35.263222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1082/2156] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:16:36.270731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea389ea1-cd44-4319-9d09-657bb071c0e2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:36.273353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ea389ea1-cd44-4319-9d09-657bb071c0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:36.294924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea389ea1-cd44-4319-9d09-657bb071c0e2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:36.302194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:36.303385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:36.303667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:36.304148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] PATH_INFO: `/v3/projects/ecb9b72f8b5c487e8b8339f487197cb1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:36.305481 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb9b72f8b5c487e8b8339f487197cb1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:36.307983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:36.382569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb9b72f8b5c487e8b8339f487197cb1', 'request_id': 'req-f9b7a47d-7610-4017-8e86-073913fb5400'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:36.394563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb9b72f8b5c487e8b8339f487197cb1', 'request_id': 'req-f9b7a47d-7610-4017-8e86-073913fb5400'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:36.404769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb9b72f8b5c487e8b8339f487197cb1', 'request_id': 'req-f9b7a47d-7610-4017-8e86-073913fb5400'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:36.425881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] The token cache is being invalidate because project ecb9b72f8b5c487e8b8339f487197cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:16:36.426301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:36.427381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f9b7a47d-7610-4017-8e86-073913fb5400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:36.430288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1075/2157] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:16:36 2022] DELETE /identity/v3/projects/ecb9b72f8b5c487e8b8339f487197cb1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:36.440060 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885c820-e478-428d-a93e-f15d364122ee None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:36.443082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885c820-e478-428d-a93e-f15d364122ee None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:36.443871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885c820-e478-428d-a93e-f15d364122ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:36.483696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1885c820-e478-428d-a93e-f15d364122ee None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:36.493538 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1885c820-e478-428d-a93e-f15d364122ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:36.529756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:36.617728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1083/2158] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:36.625970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab99d4-baac-4b93-b221-9aa9279c6811 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:36.626299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab99d4-baac-4b93-b221-9aa9279c6811 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:36.627101 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab99d4-baac-4b93-b221-9aa9279c6811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:36.648892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-91ab99d4-baac-4b93-b221-9aa9279c6811 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:36.650453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-91ab99d4-baac-4b93-b221-9aa9279c6811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:36.661119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1076/2159] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:36.666469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885c820-e478-428d-a93e-f15d364122ee None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:36.683347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885c820-e478-428d-a93e-f15d364122ee None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:36.694002 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3177c0-c7ca-491e-85d6-c9113dfe9a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:36.695160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3177c0-c7ca-491e-85d6-c9113dfe9a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:36.695860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3177c0-c7ca-491e-85d6-c9113dfe9a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:36.696368 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3177c0-c7ca-491e-85d6-c9113dfe9a52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:36.697971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3177c0-c7ca-491e-85d6-c9113dfe9a52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:16:36.698550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3177c0-c7ca-491e-85d6-c9113dfe9a52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:36.702030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3177c0-c7ca-491e-85d6-c9113dfe9a52 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:36.724945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1084/2160] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:16:36.730632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab99d4-baac-4b93-b221-9aa9279c6811 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:36.743600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab99d4-baac-4b93-b221-9aa9279c6811 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:36.754487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803973b-bb1f-4e6a-9086-64a82460c722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:36.755665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803973b-bb1f-4e6a-9086-64a82460c722 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:36.756153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803973b-bb1f-4e6a-9086-64a82460c722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:36.756635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803973b-bb1f-4e6a-9086-64a82460c722 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:36.757987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803973b-bb1f-4e6a-9086-64a82460c722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:36.761745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803973b-bb1f-4e6a-9086-64a82460c722 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:36.796589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:36.808793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1077/2161] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:16:36 2022] POST /identity/v3/projects => generated 399 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:16:36.812178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3177c0-c7ca-491e-85d6-c9113dfe9a52 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:36.834900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3177c0-c7ca-491e-85d6-c9113dfe9a52 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:36.920321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe358c3-1d66-4e34-958d-ae8d9a4d8923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:36.921474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe358c3-1d66-4e34-958d-ae8d9a4d8923 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:36.921980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe358c3-1d66-4e34-958d-ae8d9a4d8923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:36.922472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe358c3-1d66-4e34-958d-ae8d9a4d8923 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:36.923810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe358c3-1d66-4e34-958d-ae8d9a4d8923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:36.927892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe358c3-1d66-4e34-958d-ae8d9a4d8923 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:36.968548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1085/2162] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:16:36 2022] POST /identity/v3/users => generated 300 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:16:36.976124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803973b-bb1f-4e6a-9086-64a82460c722 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:37.002328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803973b-bb1f-4e6a-9086-64a82460c722 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:37.015312 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f929b22-53da-44d1-a4d8-cd4e1fbde9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:37.016660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f929b22-53da-44d1-a4d8-cd4e1fbde9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:37.017393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f929b22-53da-44d1-a4d8-cd4e1fbde9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:37.017986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f929b22-53da-44d1-a4d8-cd4e1fbde9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:37.019803 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f929b22-53da-44d1-a4d8-cd4e1fbde9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:37.023498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f929b22-53da-44d1-a4d8-cd4e1fbde9ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:37.053607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1078/2163] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:16:37.055477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe358c3-1d66-4e34-958d-ae8d9a4d8923 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:37.070387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe358c3-1d66-4e34-958d-ae8d9a4d8923 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:37.079407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565692dc-7dab-4bfc-97a1-eb12336db5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:37.080257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565692dc-7dab-4bfc-97a1-eb12336db5af admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:37.080706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565692dc-7dab-4bfc-97a1-eb12336db5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:37.081160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565692dc-7dab-4bfc-97a1-eb12336db5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:37.082311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565692dc-7dab-4bfc-97a1-eb12336db5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:37.085097 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565692dc-7dab-4bfc-97a1-eb12336db5af admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:37.103972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1086/2164] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:16:37.108362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f929b22-53da-44d1-a4d8-cd4e1fbde9ba admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:37.120374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f929b22-53da-44d1-a4d8-cd4e1fbde9ba admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:37.125968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a73150-8527-4a3c-a3fe-cd7e3e998c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:37.126928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a73150-8527-4a3c-a3fe-cd7e3e998c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:37.127218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a73150-8527-4a3c-a3fe-cd7e3e998c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:37.127542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a73150-8527-4a3c-a3fe-cd7e3e998c14 admin admin] PATH_INFO: `/v3/projects/d55410caa1174e7c80c1b6ef92a3a977/users/5645afe514e8481da6eed878a0124715/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:37.150650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a73150-8527-4a3c-a3fe-cd7e3e998c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55410caa1174e7c80c1b6ef92a3a977, user_id=5645afe514e8481da6eed878a0124715, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:37.154946 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a73150-8527-4a3c-a3fe-cd7e3e998c14 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:37.189692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1079/2165] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:16:37 2022] PUT /identity/v3/projects/d55410caa1174e7c80c1b6ef92a3a977/users/5645afe514e8481da6eed878a0124715/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:37.203006 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad8747-7588-4f84-991c-0c54187b081e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:37.203602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad8747-7588-4f84-991c-0c54187b081e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:37.204063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad8747-7588-4f84-991c-0c54187b081e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:37.253096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fdad8747-7588-4f84-991c-0c54187b081e None None] MFA Rules not processed for user `5645afe514e8481da6eed878a0124715`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:37.254384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fdad8747-7588-4f84-991c-0c54187b081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:37.358747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1087/2166] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:37.373071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a73150-8527-4a3c-a3fe-cd7e3e998c14 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:37.375148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-70a73150-8527-4a3c-a3fe-cd7e3e998c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:37.473937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a73150-8527-4a3c-a3fe-cd7e3e998c14 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:37.480068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4ea53-d425-4749-9d46-9516302e6156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:37.480921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4ea53-d425-4749-9d46-9516302e6156 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:37.482570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4ea53-d425-4749-9d46-9516302e6156 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:37.482570 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4ea53-d425-4749-9d46-9516302e6156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:37.495785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4ea53-d425-4749-9d46-9516302e6156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:37.499182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4ea53-d425-4749-9d46-9516302e6156 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:37.515852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1080/2167] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:16:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:16:47.355237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:47.443821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad8747-7588-4f84-991c-0c54187b081e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:47.508642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:47.555579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad8747-7588-4f84-991c-0c54187b081e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:47.566146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:47.567352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:47.567905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:47.568288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] PATH_INFO: `/v3/users/5645afe514e8481da6eed878a0124715` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:47.569835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] RBAC: Authorizing `identity:delete_user(user_id=5645afe514e8481da6eed878a0124715)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:47.571937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:47.698605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5645afe514e8481da6eed878a0124715'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:47.699354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5645afe514e8481da6eed878a0124715'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:47.711928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5645afe514e8481da6eed878a0124715'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:47.742339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5645afe514e8481da6eed878a0124715'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:47.752651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7cc2047c-b926-4ba3-b1b7-ec6d1d44c0af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5645afe514e8481da6eed878a0124715'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:47.755236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4ea53-d425-4749-9d46-9516302e6156 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:47.756786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c9d4ea53-d425-4749-9d46-9516302e6156 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:47.767593 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1088/2168] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:16:47 2022] DELETE /identity/v3/users/5645afe514e8481da6eed878a0124715 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:47.776547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5e63f-392d-4754-9c77-31ca8f8e074a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:47.777059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5e63f-392d-4754-9c77-31ca8f8e074a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:47.777501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5e63f-392d-4754-9c77-31ca8f8e074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:47.795514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4ea53-d425-4749-9d46-9516302e6156 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:47.802102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c3a5e63f-392d-4754-9c77-31ca8f8e074a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:47.803408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c3a5e63f-392d-4754-9c77-31ca8f8e074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:47.927601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1089/2169] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:47.928779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5e63f-392d-4754-9c77-31ca8f8e074a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:47.930309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c3a5e63f-392d-4754-9c77-31ca8f8e074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:47.950897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:47.952226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:47.952661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:47.952953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] PATH_INFO: `/v3/users/5fceafd63ed84a019fed9ba232e90ac3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:47.956319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fceafd63ed84a019fed9ba232e90ac3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:47.960623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:48.000671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:48.066979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fceafd63ed84a019fed9ba232e90ac3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:48.079342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fceafd63ed84a019fed9ba232e90ac3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:48.100517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fceafd63ed84a019fed9ba232e90ac3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:48.150967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5e63f-392d-4754-9c77-31ca8f8e074a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:48.157532 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:48.158492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fceafd63ed84a019fed9ba232e90ac3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:48.159828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-47caac7a-d984-4b4c-92ca-5c16fbf6ca42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fceafd63ed84a019fed9ba232e90ac3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:48.161112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:48.161682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:48.162172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:48.165483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:48.183829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1081/2170] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:16:47 2022] DELETE /identity/v3/users/5fceafd63ed84a019fed9ba232e90ac3 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:16:48.190581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:48.192387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7bcb4-82e4-44eb-888f-661e208184e4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:48.192732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7bcb4-82e4-44eb-888f-661e208184e4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:48.193152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:48.193813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7bcb4-82e4-44eb-888f-661e208184e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:48.213175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1090/2171] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:16:48.236744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-90f7bcb4-82e4-44eb-888f-661e208184e4 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:48.239741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-90f7bcb4-82e4-44eb-888f-661e208184e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:48.329986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1082/2172] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:48.350419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:48.352095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:48.374842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea378b-5cc7-4a10-b0a6-97e724d16c81 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:48.474126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262002a-2cee-4e9f-a7fa-d2298f789591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:48.475015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262002a-2cee-4e9f-a7fa-d2298f789591 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:48.475416 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262002a-2cee-4e9f-a7fa-d2298f789591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:48.475768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262002a-2cee-4e9f-a7fa-d2298f789591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:48.489995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262002a-2cee-4e9f-a7fa-d2298f789591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:48.492509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262002a-2cee-4e9f-a7fa-d2298f789591 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:48.516224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1091/2173] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:16:48.818398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7bcb4-82e4-44eb-888f-661e208184e4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:48.820035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-90f7bcb4-82e4-44eb-888f-661e208184e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:48.844496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7bcb4-82e4-44eb-888f-661e208184e4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:48.844496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:48.849049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:48.849049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:48.849049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] PATH_INFO: `/v3/projects/d55410caa1174e7c80c1b6ef92a3a977` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:48.849049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55410caa1174e7c80c1b6ef92a3a977)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:48.850728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:48.879816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:48.953000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55410caa1174e7c80c1b6ef92a3a977', 'request_id': 'req-98ee8bae-e2c6-406c-8458-1075ede65d45'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:48.961640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55410caa1174e7c80c1b6ef92a3a977', 'request_id': 'req-98ee8bae-e2c6-406c-8458-1075ede65d45'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:48.973301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55410caa1174e7c80c1b6ef92a3a977', 'request_id': 'req-98ee8bae-e2c6-406c-8458-1075ede65d45'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:48.997280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] The token cache is being invalidate because project d55410caa1174e7c80c1b6ef92a3a977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:16:48.997665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:48.998779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98ee8bae-e2c6-406c-8458-1075ede65d45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:49.001907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1083/2174] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:16:48 2022] DELETE /identity/v3/projects/d55410caa1174e7c80c1b6ef92a3a977 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:49.174381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262002a-2cee-4e9f-a7fa-d2298f789591 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:49.177517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d262002a-2cee-4e9f-a7fa-d2298f789591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:49.237731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262002a-2cee-4e9f-a7fa-d2298f789591 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:49.430090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:49.431432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:49.431795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:49.432323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] PATH_INFO: `/v3/projects/7e769192a87447de9f788a2fba3b4009` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:49.434860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e769192a87447de9f788a2fba3b4009)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:49.439052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:49.528074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e769192a87447de9f788a2fba3b4009', 'request_id': 'req-160417c4-3e11-4f2b-8e6e-e4b784e9208a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:49.539487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e769192a87447de9f788a2fba3b4009', 'request_id': 'req-160417c4-3e11-4f2b-8e6e-e4b784e9208a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:49.554132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e769192a87447de9f788a2fba3b4009', 'request_id': 'req-160417c4-3e11-4f2b-8e6e-e4b784e9208a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:49.561689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:49.583111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] The token cache is being invalidate because project 7e769192a87447de9f788a2fba3b4009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:16:49.584211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:49.585863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-160417c4-3e11-4f2b-8e6e-e4b784e9208a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:16:49.588674 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1092/2175] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:16:49 2022] DELETE /identity/v3/projects/7e769192a87447de9f788a2fba3b4009 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:49.595882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e4d4a-9b26-45cd-acc2-c3c2a79ad482 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:49.596338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e4d4a-9b26-45cd-acc2-c3c2a79ad482 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:49.596722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e4d4a-9b26-45cd-acc2-c3c2a79ad482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:49.633024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e57e4d4a-9b26-45cd-acc2-c3c2a79ad482 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:49.635303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e57e4d4a-9b26-45cd-acc2-c3c2a79ad482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:49.733068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1084/2176] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:49.746010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f19f26-6b53-4538-86b5-d0f9e918aa8b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:49.746579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f19f26-6b53-4538-86b5-d0f9e918aa8b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:49.747637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f19f26-6b53-4538-86b5-d0f9e918aa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:49.777307 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c8f19f26-6b53-4538-86b5-d0f9e918aa8b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:49.779877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c8f19f26-6b53-4538-86b5-d0f9e918aa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:49.792814 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1093/2177] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:49.798541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e4d4a-9b26-45cd-acc2-c3c2a79ad482 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:49.809313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e4d4a-9b26-45cd-acc2-c3c2a79ad482 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:49.814144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997aa08-156a-4305-a499-a0596d26c309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:49.814820 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997aa08-156a-4305-a499-a0596d26c309 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:49.815106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997aa08-156a-4305-a499-a0596d26c309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:49.815419 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997aa08-156a-4305-a499-a0596d26c309 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:49.816154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997aa08-156a-4305-a499-a0596d26c309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:16:49.816463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997aa08-156a-4305-a499-a0596d26c309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:49.818087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997aa08-156a-4305-a499-a0596d26c309 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:49.831211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1085/2178] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:16:49.837746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f19f26-6b53-4538-86b5-d0f9e918aa8b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:49.851525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f19f26-6b53-4538-86b5-d0f9e918aa8b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:49.860111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039c752-53ff-4153-b956-ff4e5f68de3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:49.861241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039c752-53ff-4153-b956-ff4e5f68de3d admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:49.861750 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039c752-53ff-4153-b956-ff4e5f68de3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:49.862254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039c752-53ff-4153-b956-ff4e5f68de3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:49.863644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039c752-53ff-4153-b956-ff4e5f68de3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:49.867643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039c752-53ff-4153-b956-ff4e5f68de3d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:49.910393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1094/2179] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:16:49 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:16:49.914801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997aa08-156a-4305-a499-a0596d26c309 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:49.926921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997aa08-156a-4305-a499-a0596d26c309 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:49.989913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b243f54-62da-4636-ab10-81c889f5ecf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:49.990590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243f54-62da-4636-ab10-81c889f5ecf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:49.990926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243f54-62da-4636-ab10-81c889f5ecf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:49.991296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243f54-62da-4636-ab10-81c889f5ecf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:49.992016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b243f54-62da-4636-ab10-81c889f5ecf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:49.994038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b243f54-62da-4636-ab10-81c889f5ecf4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:50.017046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1086/2180] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:16:49 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:16:50.024740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039c752-53ff-4153-b956-ff4e5f68de3d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:50.037012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039c752-53ff-4153-b956-ff4e5f68de3d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:50.042711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287b01a-4d87-4f9b-b590-577e89a16d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:50.044078 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287b01a-4d87-4f9b-b590-577e89a16d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:50.044420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287b01a-4d87-4f9b-b590-577e89a16d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:50.044703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287b01a-4d87-4f9b-b590-577e89a16d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:50.045592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287b01a-4d87-4f9b-b590-577e89a16d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:50.047576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287b01a-4d87-4f9b-b590-577e89a16d59 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:50.063247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1095/2181] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:16:50.067875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b243f54-62da-4636-ab10-81c889f5ecf4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:50.080471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b243f54-62da-4636-ab10-81c889f5ecf4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:50.088676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecd44d-1f3e-47bb-b510-d8ebde827988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:50.090027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd44d-1f3e-47bb-b510-d8ebde827988 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:50.091056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd44d-1f3e-47bb-b510-d8ebde827988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:50.091605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd44d-1f3e-47bb-b510-d8ebde827988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:50.092947 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecd44d-1f3e-47bb-b510-d8ebde827988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:50.095948 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecd44d-1f3e-47bb-b510-d8ebde827988 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:50.113028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1087/2182] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:16:50.119638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287b01a-4d87-4f9b-b590-577e89a16d59 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:50.131922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287b01a-4d87-4f9b-b590-577e89a16d59 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:50.136796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec57a7-336a-4c12-ac20-9b5e8d264c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:50.138601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec57a7-336a-4c12-ac20-9b5e8d264c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:50.138601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec57a7-336a-4c12-ac20-9b5e8d264c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:50.138601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec57a7-336a-4c12-ac20-9b5e8d264c5b admin admin] PATH_INFO: `/v3/projects/f22c473beee4486d977ff684bbbf3e37/users/75b52d157ef84e6e9a0f7867eb5ed4cc/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:50.154148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec57a7-336a-4c12-ac20-9b5e8d264c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22c473beee4486d977ff684bbbf3e37, user_id=75b52d157ef84e6e9a0f7867eb5ed4cc, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:50.156899 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec57a7-336a-4c12-ac20-9b5e8d264c5b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:50.173291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1096/2183] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:16:50 2022] PUT /identity/v3/projects/f22c473beee4486d977ff684bbbf3e37/users/75b52d157ef84e6e9a0f7867eb5ed4cc/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:16:50.186352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421c1a2-c16a-48bc-a3c2-688564ff9cd4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:50.186858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421c1a2-c16a-48bc-a3c2-688564ff9cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:50.187345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421c1a2-c16a-48bc-a3c2-688564ff9cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:50.213844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:16:50.232824 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8421c1a2-c16a-48bc-a3c2-688564ff9cd4 None None] MFA Rules not processed for user `75b52d157ef84e6e9a0f7867eb5ed4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:50.234100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8421c1a2-c16a-48bc-a3c2-688564ff9cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:50.341415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1088/2184] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:16:50.358417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec57a7-336a-4c12-ac20-9b5e8d264c5b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:16:50.361162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-77ec57a7-336a-4c12-ac20-9b5e8d264c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:50.454427 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec57a7-336a-4c12-ac20-9b5e8d264c5b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:16:50.464787 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5888f-d22b-4f75-b5c7-9b09292ec50a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:16:50.466081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5888f-d22b-4f75-b5c7-9b09292ec50a service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:50.466441 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5888f-d22b-4f75-b5c7-9b09292ec50a service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:50.466711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5888f-d22b-4f75-b5c7-9b09292ec50a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:50.491817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5888f-d22b-4f75-b5c7-9b09292ec50a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:16:50.495723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5888f-d22b-4f75-b5c7-9b09292ec50a service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:16:50.511544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1097/2185] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:16:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:16:52.441391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c84dc1-30f3-47fc-b922-d226f15c15bd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:16:52.442211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c84dc1-30f3-47fc-b922-d226f15c15bd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:16:52.442673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c84dc1-30f3-47fc-b922-d226f15c15bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:16:52.463568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-51c84dc1-30f3-47fc-b922-d226f15c15bd None None] MFA Rules not processed for user `7c2c0270618240de8fd9290d01a7413a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:16:52.465146 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-51c84dc1-30f3-47fc-b922-d226f15c15bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:16:52.545756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1089/2186] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:16:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:17:00.504248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:17:02.530143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=53 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:17:05.781345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5888f-d22b-4f75-b5c7-9b09292ec50a service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:05.784134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-67b5888f-d22b-4f75-b5c7-9b09292ec50a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:05.936217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5888f-d22b-4f75-b5c7-9b09292ec50a service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:05.945204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee8dff-ad53-45c0-9716-e393a9797f70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:05.946338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee8dff-ad53-45c0-9716-e393a9797f70 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:05.946822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee8dff-ad53-45c0-9716-e393a9797f70 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:05.947327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee8dff-ad53-45c0-9716-e393a9797f70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:05.950213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-00ee8dff-ad53-45c0-9716-e393a9797f70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:05.970805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee8dff-ad53-45c0-9716-e393a9797f70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:05.972798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee8dff-ad53-45c0-9716-e393a9797f70 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:05.990319 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1098/2187] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:17:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:17:07.422622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c84dc1-30f3-47fc-b922-d226f15c15bd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:07.424639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-51c84dc1-30f3-47fc-b922-d226f15c15bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:07.510236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c84dc1-30f3-47fc-b922-d226f15c15bd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:07.515207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f5d8b-9dda-4d85-95a7-362e235290b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:07.515932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f5d8b-9dda-4d85-95a7-362e235290b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:07.516223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f5d8b-9dda-4d85-95a7-362e235290b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:07.516524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f5d8b-9dda-4d85-95a7-362e235290b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:07.518596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b42f5d8b-9dda-4d85-95a7-362e235290b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:07.592694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f5d8b-9dda-4d85-95a7-362e235290b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:07.593170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f5d8b-9dda-4d85-95a7-362e235290b6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:07.612766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1090/2188] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:17:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:17:15.981423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=32 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:17:17.600297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=50 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:17:36.232894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee8dff-ad53-45c0-9716-e393a9797f70 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:36.323989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee8dff-ad53-45c0-9716-e393a9797f70 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:36.330745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:36.332359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:36.332822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:36.333295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] PATH_INFO: `/v3/users/75b52d157ef84e6e9a0f7867eb5ed4cc` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:36.334991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75b52d157ef84e6e9a0f7867eb5ed4cc)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:36.337208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:36.404663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75b52d157ef84e6e9a0f7867eb5ed4cc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:36.405547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75b52d157ef84e6e9a0f7867eb5ed4cc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:36.422689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75b52d157ef84e6e9a0f7867eb5ed4cc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:36.437710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75b52d157ef84e6e9a0f7867eb5ed4cc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:36.445642 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75b52d157ef84e6e9a0f7867eb5ed4cc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:36.454527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1099/2189] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:17:36 2022] DELETE /identity/v3/users/75b52d157ef84e6e9a0f7867eb5ed4cc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:17:36.459496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b59f6-ac64-4293-9278-2be1cd4795ce None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:36.460052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b59f6-ac64-4293-9278-2be1cd4795ce None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:36.460207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b59f6-ac64-4293-9278-2be1cd4795ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:36.480230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f46b59f6-ac64-4293-9278-2be1cd4795ce None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:17:36.481462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f46b59f6-ac64-4293-9278-2be1cd4795ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:36.550078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1091/2190] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:17:36.562575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:36.564382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:36.634648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796304-afec-41f7-b456-cf157e09cbd2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:36.640573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b00f7-3ac1-41a1-a252-bdaaf3e7df7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:36.641282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b00f7-3ac1-41a1-a252-bdaaf3e7df7b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:36.641579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b00f7-3ac1-41a1-a252-bdaaf3e7df7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:36.641887 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b00f7-3ac1-41a1-a252-bdaaf3e7df7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:36.653927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b00f7-3ac1-41a1-a252-bdaaf3e7df7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:36.655810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b00f7-3ac1-41a1-a252-bdaaf3e7df7b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:36.668120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1100/2191] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:17:37.033699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b59f6-ac64-4293-9278-2be1cd4795ce None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:37.035229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f46b59f6-ac64-4293-9278-2be1cd4795ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:37.050243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b59f6-ac64-4293-9278-2be1cd4795ce None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:37.055626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:37.056511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.056818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.057108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] PATH_INFO: `/v3/projects/f22c473beee4486d977ff684bbbf3e37` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.058358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] RBAC: Authorizing `identity:delete_project(project_id=f22c473beee4486d977ff684bbbf3e37)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:37.060364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:37.128089 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f22c473beee4486d977ff684bbbf3e37', 'request_id': 'req-3336df88-00af-4445-aca8-840f3cb935da'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:37.134863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f22c473beee4486d977ff684bbbf3e37', 'request_id': 'req-3336df88-00af-4445-aca8-840f3cb935da'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:37.141207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f22c473beee4486d977ff684bbbf3e37', 'request_id': 'req-3336df88-00af-4445-aca8-840f3cb935da'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:37.156715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] The token cache is being invalidate because project f22c473beee4486d977ff684bbbf3e37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:17:37.157190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:37.157896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3336df88-00af-4445-aca8-840f3cb935da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:37.159865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1092/2192] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:17:37 2022] DELETE /identity/v3/projects/f22c473beee4486d977ff684bbbf3e37 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:17:37.165349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04448df-6960-4106-983b-69f3bd90f3fc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.165860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04448df-6960-4106-983b-69f3bd90f3fc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.166265 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04448df-6960-4106-983b-69f3bd90f3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.187723 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d04448df-6960-4106-983b-69f3bd90f3fc None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:17:37.189032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d04448df-6960-4106-983b-69f3bd90f3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:37.208736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:17:37.278222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1101/2193] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:17:37.286781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12125db-8f64-4a23-b448-051f81344471 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.287120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12125db-8f64-4a23-b448-051f81344471 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.287489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12125db-8f64-4a23-b448-051f81344471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.308018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c12125db-8f64-4a23-b448-051f81344471 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:17:37.309409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c12125db-8f64-4a23-b448-051f81344471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:37.318168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1093/2194] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:17:37.323135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04448df-6960-4106-983b-69f3bd90f3fc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:37.334676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04448df-6960-4106-983b-69f3bd90f3fc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:37.340288 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bdd3a-00e7-4c7a-8c57-7a879d6e3a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:37.340966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bdd3a-00e7-4c7a-8c57-7a879d6e3a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.341254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bdd3a-00e7-4c7a-8c57-7a879d6e3a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.341535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bdd3a-00e7-4c7a-8c57-7a879d6e3a19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.342280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bdd3a-00e7-4c7a-8c57-7a879d6e3a19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:17:37.342594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bdd3a-00e7-4c7a-8c57-7a879d6e3a19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:37.344248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bdd3a-00e7-4c7a-8c57-7a879d6e3a19 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:37.357897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1102/2195] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:17:37.362912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12125db-8f64-4a23-b448-051f81344471 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:37.373179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12125db-8f64-4a23-b448-051f81344471 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:37.378391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0857e0-c961-4999-8c28-45c2f4f23390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:37.379208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0857e0-c961-4999-8c28-45c2f4f23390 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.379647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0857e0-c961-4999-8c28-45c2f4f23390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.380090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0857e0-c961-4999-8c28-45c2f4f23390 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.380961 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0857e0-c961-4999-8c28-45c2f4f23390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:37.383214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0857e0-c961-4999-8c28-45c2f4f23390 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:37.413853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1094/2196] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:17:37 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:17:37.420669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bdd3a-00e7-4c7a-8c57-7a879d6e3a19 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:37.431714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bdd3a-00e7-4c7a-8c57-7a879d6e3a19 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:37.531282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fa2b1-c2e5-478e-b2f5-e9caa771a434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:37.532342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fa2b1-c2e5-478e-b2f5-e9caa771a434 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.532821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fa2b1-c2e5-478e-b2f5-e9caa771a434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.533296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fa2b1-c2e5-478e-b2f5-e9caa771a434 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.534577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fa2b1-c2e5-478e-b2f5-e9caa771a434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:37.538225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fa2b1-c2e5-478e-b2f5-e9caa771a434 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:37.570119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1103/2197] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:17:37 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:17:37.576439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0857e0-c961-4999-8c28-45c2f4f23390 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:37.597403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0857e0-c961-4999-8c28-45c2f4f23390 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:37.604026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7b425-e8be-46c3-87ef-bad57e4b5cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:37.604982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7b425-e8be-46c3-87ef-bad57e4b5cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.605270 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7b425-e8be-46c3-87ef-bad57e4b5cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.605537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7b425-e8be-46c3-87ef-bad57e4b5cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.606320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7b425-e8be-46c3-87ef-bad57e4b5cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:37.608412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7b425-e8be-46c3-87ef-bad57e4b5cd4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:37.620917 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1095/2198] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:17:37.625544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fa2b1-c2e5-478e-b2f5-e9caa771a434 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:37.642021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fa2b1-c2e5-478e-b2f5-e9caa771a434 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:37.647166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30ac1b-b057-4fe1-aadc-4e6a25bc1d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:37.647867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30ac1b-b057-4fe1-aadc-4e6a25bc1d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.648155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30ac1b-b057-4fe1-aadc-4e6a25bc1d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.648560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30ac1b-b057-4fe1-aadc-4e6a25bc1d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.649209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30ac1b-b057-4fe1-aadc-4e6a25bc1d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:37.650898 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30ac1b-b057-4fe1-aadc-4e6a25bc1d0c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:37.665316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1104/2199] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:17:37.669782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7b425-e8be-46c3-87ef-bad57e4b5cd4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:37.683750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7b425-e8be-46c3-87ef-bad57e4b5cd4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:37.692184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f6963-e771-4985-83b1-0e6c9c578893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:37.693769 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f6963-e771-4985-83b1-0e6c9c578893 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.694279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f6963-e771-4985-83b1-0e6c9c578893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.694794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f6963-e771-4985-83b1-0e6c9c578893 admin admin] PATH_INFO: `/v3/projects/bad1cf800f59414daa01834aea42a7e1/users/6d10589e113449e298f16423b0b230b4/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.714725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f6963-e771-4985-83b1-0e6c9c578893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad1cf800f59414daa01834aea42a7e1, user_id=6d10589e113449e298f16423b0b230b4, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:37.717887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f6963-e771-4985-83b1-0e6c9c578893 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:37.733073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1096/2200] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:17:37 2022] PUT /identity/v3/projects/bad1cf800f59414daa01834aea42a7e1/users/6d10589e113449e298f16423b0b230b4/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:17:37.805034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996f1a6-1766-448a-9a83-a4c30cc45228 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:37.805556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996f1a6-1766-448a-9a83-a4c30cc45228 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:37.806324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996f1a6-1766-448a-9a83-a4c30cc45228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:37.848842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6996f1a6-1766-448a-9a83-a4c30cc45228 None None] MFA Rules not processed for user `6d10589e113449e298f16423b0b230b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:17:37.850498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6996f1a6-1766-448a-9a83-a4c30cc45228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:37.940865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1105/2201] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:17:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:17:37.954096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f6963-e771-4985-83b1-0e6c9c578893 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:37.955984 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-750f6963-e771-4985-83b1-0e6c9c578893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:37.983289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:17:38.035663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f6963-e771-4985-83b1-0e6c9c578893 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:38.039836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863d464-25ce-404f-883b-ed57b73c30da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:38.040502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863d464-25ce-404f-883b-ed57b73c30da service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:38.040798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863d464-25ce-404f-883b-ed57b73c30da service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:38.041083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863d464-25ce-404f-883b-ed57b73c30da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:38.050896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863d464-25ce-404f-883b-ed57b73c30da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:38.052694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863d464-25ce-404f-883b-ed57b73c30da service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:38.064063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1097/2202] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:17:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:17:38.099497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996f1a6-1766-448a-9a83-a4c30cc45228 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:38.121556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:17:38.191098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996f1a6-1766-448a-9a83-a4c30cc45228 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:38.198628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:38.200172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:38.200676 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:38.201180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] PATH_INFO: `/v3/users/6d10589e113449e298f16423b0b230b4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:38.203222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d10589e113449e298f16423b0b230b4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:38.206889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:38.266508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d10589e113449e298f16423b0b230b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:38.267214 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d10589e113449e298f16423b0b230b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:38.275812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d10589e113449e298f16423b0b230b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:38.292793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d10589e113449e298f16423b0b230b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:38.301171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d10589e113449e298f16423b0b230b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:38.310742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1106/2203] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:17:38 2022] DELETE /identity/v3/users/6d10589e113449e298f16423b0b230b4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:17:38.319740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42136565-3d2b-4bec-9d3d-1d4b56b52bdf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:38.320203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42136565-3d2b-4bec-9d3d-1d4b56b52bdf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:38.320534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42136565-3d2b-4bec-9d3d-1d4b56b52bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:38.346749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-42136565-3d2b-4bec-9d3d-1d4b56b52bdf None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:17:38.348836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-42136565-3d2b-4bec-9d3d-1d4b56b52bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:38.447048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1098/2204] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:17:38.459491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:38.461163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:38.534289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794493cf-6c59-4cf8-b10c-8336e9d96b30 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:38.538287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f581ea7-13b2-492c-89b4-4b04eb6c7ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:38.539037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f581ea7-13b2-492c-89b4-4b04eb6c7ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:38.539359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f581ea7-13b2-492c-89b4-4b04eb6c7ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:38.539654 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f581ea7-13b2-492c-89b4-4b04eb6c7ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:38.550454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f581ea7-13b2-492c-89b4-4b04eb6c7ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:38.551920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f581ea7-13b2-492c-89b4-4b04eb6c7ef5 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:38.562291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1107/2205] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:17:38.902199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42136565-3d2b-4bec-9d3d-1d4b56b52bdf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:17:38.904495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-42136565-3d2b-4bec-9d3d-1d4b56b52bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:17:38.926075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42136565-3d2b-4bec-9d3d-1d4b56b52bdf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:17:38.933637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:17:38.934935 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:17:38.935487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:17:38.936114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] PATH_INFO: `/v3/projects/bad1cf800f59414daa01834aea42a7e1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:17:38.937993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad1cf800f59414daa01834aea42a7e1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:17:38.941728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:17:39.005960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad1cf800f59414daa01834aea42a7e1', 'request_id': 'req-2c506bdb-9a88-47d9-a4df-26674d39d00d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:39.012891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad1cf800f59414daa01834aea42a7e1', 'request_id': 'req-2c506bdb-9a88-47d9-a4df-26674d39d00d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:39.019025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad1cf800f59414daa01834aea42a7e1', 'request_id': 'req-2c506bdb-9a88-47d9-a4df-26674d39d00d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:39.033827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] The token cache is being invalidate because project bad1cf800f59414daa01834aea42a7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:17:39.034114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:39.034765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:17:39.036390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1099/2206] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:17:38 2022] DELETE /identity/v3/projects/bad1cf800f59414daa01834aea42a7e1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:17:48.557746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:17:49.030619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:18:06.736594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51feb39a-4c90-4128-a6b4-27c34924410b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:06.737154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51feb39a-4c90-4128-a6b4-27c34924410b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:06.737316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51feb39a-4c90-4128-a6b4-27c34924410b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:06.765389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-51feb39a-4c90-4128-a6b4-27c34924410b None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:18:06.766639 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-51feb39a-4c90-4128-a6b4-27c34924410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:06.849509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1108/2207] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:18:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:18:06.915671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:18:06.918026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:07.048728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c506bdb-9a88-47d9-a4df-26674d39d00d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:18:07.054839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ac58e-6800-4eed-8465-22854d497a08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:18:07.055797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ac58e-6800-4eed-8465-22854d497a08 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:07.057299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ac58e-6800-4eed-8465-22854d497a08 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:07.057299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ac58e-6800-4eed-8465-22854d497a08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:07.074523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ac58e-6800-4eed-8465-22854d497a08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:18:07.076334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ac58e-6800-4eed-8465-22854d497a08 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:18:07.088360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1100/2208] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:18:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:18:16.841708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=25 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:18:17.038395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ffcbb-54b7-41a6-a600-e3fbc4c8684a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:17.038694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ffcbb-54b7-41a6-a600-e3fbc4c8684a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:17.038978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ffcbb-54b7-41a6-a600-e3fbc4c8684a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:17.065395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7b9ffcbb-54b7-41a6-a600-e3fbc4c8684a None None] MFA Rules not processed for user `691877dce1024aeda61629073d74602b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:18:17.066982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7b9ffcbb-54b7-41a6-a600-e3fbc4c8684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:17.082637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=32 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:18:17.165354 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1109/2209] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:18:25.288562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b3bc6-d85c-45a6-8fe1-ddf0ec6344c2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:25.289560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b3bc6-d85c-45a6-8fe1-ddf0ec6344c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:25.289560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b3bc6-d85c-45a6-8fe1-ddf0ec6344c2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:25.292240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1101/2210] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:18:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:18:25.300146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babac8fc-3755-4f72-8ee9-df8fb6f070a7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:25.300832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babac8fc-3755-4f72-8ee9-df8fb6f070a7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:25.301133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babac8fc-3755-4f72-8ee9-df8fb6f070a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:25.335663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-babac8fc-3755-4f72-8ee9-df8fb6f070a7 None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:18:25.337454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-babac8fc-3755-4f72-8ee9-df8fb6f070a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:25.435819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1110/2211] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:18:25.449422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b3bc6-d85c-45a6-8fe1-ddf0ec6344c2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:18:25.451359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-985b3bc6-d85c-45a6-8fe1-ddf0ec6344c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:25.526264 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b3bc6-d85c-45a6-8fe1-ddf0ec6344c2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:18:25.530268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346e342-969f-4542-a70e-c3f26d19114d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92fc25f9ad7c49c48a909d2d92ea0a66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:18:25.531103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346e342-969f-4542-a70e-c3f26d19114d service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:25.531413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346e342-969f-4542-a70e-c3f26d19114d service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:25.531704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346e342-969f-4542-a70e-c3f26d19114d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:25.549318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346e342-969f-4542-a70e-c3f26d19114d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:18:25.550833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346e342-969f-4542-a70e-c3f26d19114d service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:18:25.569275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1102/2212] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:18:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:18:25.663073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342141a2-c43b-425a-8cfe-906a369d7141 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:25.663467 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342141a2-c43b-425a-8cfe-906a369d7141 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:25.663848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342141a2-c43b-425a-8cfe-906a369d7141 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:25.666453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1111/2213] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:18:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:18:25.672592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1b81e-23a8-46f9-8460-d6c3b45c9bd5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:25.672886 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1b81e-23a8-46f9-8460-d6c3b45c9bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:25.673165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1b81e-23a8-46f9-8460-d6c3b45c9bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:25.699016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-28f1b81e-23a8-46f9-8460-d6c3b45c9bd5 None None] MFA Rules not processed for user `0a0ce1804b514b42845ab33c7bfee309`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:18:25.700315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28f1b81e-23a8-46f9-8460-d6c3b45c9bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:25.773092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1103/2214] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:18:25.787206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342141a2-c43b-425a-8cfe-906a369d7141 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:18:25.789197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-342141a2-c43b-425a-8cfe-906a369d7141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:25.803688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342141a2-c43b-425a-8cfe-906a369d7141 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:18:25.807418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07caa3fe-ff3e-4997-af5d-dc5395733fb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:18:25.808075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caa3fe-ff3e-4997-af5d-dc5395733fb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:25.808363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caa3fe-ff3e-4997-af5d-dc5395733fb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:25.808641 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caa3fe-ff3e-4997-af5d-dc5395733fb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:25.816628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07caa3fe-ff3e-4997-af5d-dc5395733fb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:18:25.818097 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07caa3fe-ff3e-4997-af5d-dc5395733fb3 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:18:25.827027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1112/2215] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:18:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:18:25.918983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162bbd9-df39-4ccd-ae72-e5795c3a9be1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:25.919191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162bbd9-df39-4ccd-ae72-e5795c3a9be1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:25.919525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162bbd9-df39-4ccd-ae72-e5795c3a9be1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:25.920981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1104/2216] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:18:25.926399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769a7e8-8d61-45b2-8d75-68a6d18c08b4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:25.926697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769a7e8-8d61-45b2-8d75-68a6d18c08b4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:25.926983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769a7e8-8d61-45b2-8d75-68a6d18c08b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:25.946373 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7769a7e8-8d61-45b2-8d75-68a6d18c08b4 None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:18:25.947666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7769a7e8-8d61-45b2-8d75-68a6d18c08b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:25.957403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1113/2217] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:18:25.969080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162bbd9-df39-4ccd-ae72-e5795c3a9be1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:18:25.983474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162bbd9-df39-4ccd-ae72-e5795c3a9be1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:18:25.987408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5f367-eeed-472f-bcf4-11ebfc7b0bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:18:25.988067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5f367-eeed-472f-bcf4-11ebfc7b0bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:25.988356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5f367-eeed-472f-bcf4-11ebfc7b0bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:25.988640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5f367-eeed-472f-bcf4-11ebfc7b0bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:26.007884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5f367-eeed-472f-bcf4-11ebfc7b0bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:18:26.009378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5f367-eeed-472f-bcf4-11ebfc7b0bd9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:18:26.024751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1105/2218] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:18:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:18:28.963874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f962fc-97dd-4c19-8333-d395df1941fb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:28.964424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f962fc-97dd-4c19-8333-d395df1941fb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:28.964871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f962fc-97dd-4c19-8333-d395df1941fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:28.992813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-83f962fc-97dd-4c19-8333-d395df1941fb None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:18:28.994909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-83f962fc-97dd-4c19-8333-d395df1941fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:29.006168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1114/2219] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:18:29.020557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5f367-eeed-472f-bcf4-11ebfc7b0bd9 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:18:29.033862 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5f367-eeed-472f-bcf4-11ebfc7b0bd9 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:18:29.039528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aedb44-092f-425d-8571-8ef0c7822720 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:18:29.041000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aedb44-092f-425d-8571-8ef0c7822720 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:29.041619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aedb44-092f-425d-8571-8ef0c7822720 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:29.042053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aedb44-092f-425d-8571-8ef0c7822720 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:29.054471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aedb44-092f-425d-8571-8ef0c7822720 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:18:29.056558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aedb44-092f-425d-8571-8ef0c7822720 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:18:29.088179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1106/2220] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:18:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:18:38.986180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=68 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:18:39.062356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=61 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:18:44.393453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151987ef-a1ac-4300-88d1-62fd4d6cec5a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:44.394876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151987ef-a1ac-4300-88d1-62fd4d6cec5a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:44.395173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151987ef-a1ac-4300-88d1-62fd4d6cec5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:44.422091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-151987ef-a1ac-4300-88d1-62fd4d6cec5a None None] MFA Rules not processed for user `7c2c0270618240de8fd9290d01a7413a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:18:44.423726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-151987ef-a1ac-4300-88d1-62fd4d6cec5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:44.510974 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1115/2221] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:18:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:18:52.753019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2491117-04fe-4a33-9114-b842afb5751e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:52.753745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2491117-04fe-4a33-9114-b842afb5751e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:52.753904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2491117-04fe-4a33-9114-b842afb5751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:52.788475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d2491117-04fe-4a33-9114-b842afb5751e None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:18:52.791022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d2491117-04fe-4a33-9114-b842afb5751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:18:52.801141 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1107/2222] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 22 11:18:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:18:52.818350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151987ef-a1ac-4300-88d1-62fd4d6cec5a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:18:52.834184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151987ef-a1ac-4300-88d1-62fd4d6cec5a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:18:52.843113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc026-08b1-45c1-8f79-5a15572d9b65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:18:52.844268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc026-08b1-45c1-8f79-5a15572d9b65 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:18:52.844774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc026-08b1-45c1-8f79-5a15572d9b65 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:18:52.845270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc026-08b1-45c1-8f79-5a15572d9b65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:18:52.859136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cc026-08b1-45c1-8f79-5a15572d9b65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:18:52.862063 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cc026-08b1-45c1-8f79-5a15572d9b65 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:18:52.877580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1116/2223] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:18:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:19:02.780676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:19:02.872077 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=31 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:19:25.088692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2491117-04fe-4a33-9114-b842afb5751e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:25.090250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d2491117-04fe-4a33-9114-b842afb5751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:25.102699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2491117-04fe-4a33-9114-b842afb5751e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:25.106420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e348cc-2c68-486c-983c-d020de5fd866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:25.107075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e348cc-2c68-486c-983c-d020de5fd866 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:25.107395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e348cc-2c68-486c-983c-d020de5fd866 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:25.107676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e348cc-2c68-486c-983c-d020de5fd866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:25.109691 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-56e348cc-2c68-486c-983c-d020de5fd866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:25.125496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e348cc-2c68-486c-983c-d020de5fd866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:25.127144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e348cc-2c68-486c-983c-d020de5fd866 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:25.139871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1108/2224] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:19:25.695242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc026-08b1-45c1-8f79-5a15572d9b65 service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:25.698178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3a4cc026-08b1-45c1-8f79-5a15572d9b65 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:25.747527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc026-08b1-45c1-8f79-5a15572d9b65 service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:25.765512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:25.766210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:25.766506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:25.766789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:25.772098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:25.798503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:25.802281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:25.821311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1117/2225] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:19:25.932795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e348cc-2c68-486c-983c-d020de5fd866 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:25.950022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e348cc-2c68-486c-983c-d020de5fd866 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:25.969015 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:25.970009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:25.970376 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:25.970694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:25.981793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:26.006101 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:26.008019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:26.023013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1109/2226] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:19:29.696745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:29.698301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:29.757106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744d8e3-9443-465b-aec2-3dfe3d4d52dd service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:29.760955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7315176-0a15-4b84-99ea-1e39ad9abefa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:29.761620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7315176-0a15-4b84-99ea-1e39ad9abefa service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:29.761892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7315176-0a15-4b84-99ea-1e39ad9abefa service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:29.762172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7315176-0a15-4b84-99ea-1e39ad9abefa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:29.764126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e7315176-0a15-4b84-99ea-1e39ad9abefa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:29.779005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7315176-0a15-4b84-99ea-1e39ad9abefa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:29.780642 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7315176-0a15-4b84-99ea-1e39ad9abefa service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:29.794503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1118/2227] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:19:30.915594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:30.917151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:30.929647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60b3b9-236a-4d94-af3b-10d27af0276c service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:30.934207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00ea73-8701-401a-a1e3-24602f183792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:30.934869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00ea73-8701-401a-a1e3-24602f183792 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:30.935154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00ea73-8701-401a-a1e3-24602f183792 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:30.935458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00ea73-8701-401a-a1e3-24602f183792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:30.937356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bc00ea73-8701-401a-a1e3-24602f183792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:31.004583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00ea73-8701-401a-a1e3-24602f183792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:31.006039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00ea73-8701-401a-a1e3-24602f183792 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:31.018157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1110/2228] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:19:31.958450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7315176-0a15-4b84-99ea-1e39ad9abefa service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:31.974911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7315176-0a15-4b84-99ea-1e39ad9abefa service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:31.982321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:31.983118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:31.983653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:31.983997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:31.986079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:32.000357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:32.003422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:32.018972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1119/2229] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:19:32.627208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00ea73-8701-401a-a1e3-24602f183792 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:32.639943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00ea73-8701-401a-a1e3-24602f183792 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:32.646558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a33a3-82b9-41bc-83b6-9d96fa004e5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:32.648251 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a33a3-82b9-41bc-83b6-9d96fa004e5a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:32.648582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a33a3-82b9-41bc-83b6-9d96fa004e5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:32.648864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a33a3-82b9-41bc-83b6-9d96fa004e5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:32.651450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7d0a33a3-82b9-41bc-83b6-9d96fa004e5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:32.673036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a33a3-82b9-41bc-83b6-9d96fa004e5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:32.675709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a33a3-82b9-41bc-83b6-9d96fa004e5a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:32.686788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1111/2230] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:19:37.297342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:37.299196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:37.320291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a6d64-95b4-495e-9069-10afc26c9f03 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:37.327711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c8e6-248b-4a4d-8105-eb4c1ad2b9b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:37.328724 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c8e6-248b-4a4d-8105-eb4c1ad2b9b6 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:37.329485 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c8e6-248b-4a4d-8105-eb4c1ad2b9b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:37.330105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c8e6-248b-4a4d-8105-eb4c1ad2b9b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:37.333523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d4f0c8e6-248b-4a4d-8105-eb4c1ad2b9b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:37.355405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0c8e6-248b-4a4d-8105-eb4c1ad2b9b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:37.357638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0c8e6-248b-4a4d-8105-eb4c1ad2b9b6 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:37.373728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1120/2231] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:19:39.036268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8a98d-5954-4262-97cd-82f56afe2067 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:39.037036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8a98d-5954-4262-97cd-82f56afe2067 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:39.037283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8a98d-5954-4262-97cd-82f56afe2067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:39.057201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5dd8a98d-5954-4262-97cd-82f56afe2067 None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:19:39.058511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5dd8a98d-5954-4262-97cd-82f56afe2067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:39.069554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1112/2232] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 22 11:19:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:19:39.093844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c8e6-248b-4a4d-8105-eb4c1ad2b9b6 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:39.107581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c8e6-248b-4a4d-8105-eb4c1ad2b9b6 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:39.113299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bced1-4ecb-4219-af9f-3218be23d9b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:39.113995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bced1-4ecb-4219-af9f-3218be23d9b8 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:39.115771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bced1-4ecb-4219-af9f-3218be23d9b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:39.115771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bced1-4ecb-4219-af9f-3218be23d9b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:39.124970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bced1-4ecb-4219-af9f-3218be23d9b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:39.126832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bced1-4ecb-4219-af9f-3218be23d9b8 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:39.139443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1121/2233] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:19:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:19:46.090912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64237d8-cfcb-44c5-bdce-b51f01d5c2a0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:46.091423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64237d8-cfcb-44c5-bdce-b51f01d5c2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:46.091508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64237d8-cfcb-44c5-bdce-b51f01d5c2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:46.119110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c64237d8-cfcb-44c5-bdce-b51f01d5c2a0 None None] MFA Rules not processed for user `da75b78b267f47e8aaaafb41e3acb61a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:19:46.120404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c64237d8-cfcb-44c5-bdce-b51f01d5c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:46.130640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1113/2234] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:19:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:19:46.556433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bced1-4ecb-4219-af9f-3218be23d9b8 service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:46.568685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bced1-4ecb-4219-af9f-3218be23d9b8 service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:46.574646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:46.575995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:46.576510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:46.576976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:46.579208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:46.593633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:46.596102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:46.617075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1122/2235] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:19:47.037320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64237d8-cfcb-44c5-bdce-b51f01d5c2a0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:47.040554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c64237d8-cfcb-44c5-bdce-b51f01d5c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:47.144007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64237d8-cfcb-44c5-bdce-b51f01d5c2a0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:47.149707 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cea5d4-8424-4336-93d7-05f615027f28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:47.150462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cea5d4-8424-4336-93d7-05f615027f28 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:47.150821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cea5d4-8424-4336-93d7-05f615027f28 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:47.151173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cea5d4-8424-4336-93d7-05f615027f28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:47.153200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-96cea5d4-8424-4336-93d7-05f615027f28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:47.168901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cea5d4-8424-4336-93d7-05f615027f28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:47.171508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cea5d4-8424-4336-93d7-05f615027f28 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:47.187169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1114/2236] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:19:49.625514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:49.628475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:49.705637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666af92c-b8c1-4acb-9475-cfca745f4179 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:49.711796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:49.712732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:49.713019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:49.713299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] PATH_INFO: `/v3/users/e908d2b09dfc4a139d9c27f52580e15b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:49.714549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e908d2b09dfc4a139d9c27f52580e15b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:49.716587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:49.775084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:19:49.788593 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e908d2b09dfc4a139d9c27f52580e15b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:49.789281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e908d2b09dfc4a139d9c27f52580e15b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:49.799456 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e908d2b09dfc4a139d9c27f52580e15b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:49.817139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e908d2b09dfc4a139d9c27f52580e15b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:49.826106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e908d2b09dfc4a139d9c27f52580e15b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:49.832531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1123/2237] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:19:49 2022] DELETE /identity/v3/users/e908d2b09dfc4a139d9c27f52580e15b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:19:50.030622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cea5d4-8424-4336-93d7-05f615027f28 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:50.056910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cea5d4-8424-4336-93d7-05f615027f28 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:50.103621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:19:50.142155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:50.143043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:50.143374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:50.143649 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] PATH_INFO: `/v3/projects/a84fa2f397c54be1b409d8be5ebc8491` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:50.144913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84fa2f397c54be1b409d8be5ebc8491)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:50.146852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:50.214366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84fa2f397c54be1b409d8be5ebc8491', 'request_id': 'req-81d039ba-21d2-45fa-90d8-19549efb16a1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.221573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84fa2f397c54be1b409d8be5ebc8491', 'request_id': 'req-81d039ba-21d2-45fa-90d8-19549efb16a1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.228056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84fa2f397c54be1b409d8be5ebc8491', 'request_id': 'req-81d039ba-21d2-45fa-90d8-19549efb16a1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.251332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] The token cache is being invalidate because project a84fa2f397c54be1b409d8be5ebc8491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:19:50.251332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.252276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.254210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1115/2238] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:19:50 2022] DELETE /identity/v3/projects/a84fa2f397c54be1b409d8be5ebc8491 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:19:50.258696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:50.260400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:50.277728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d9eec-b472-4a61-9229-b1b54c77ab88 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:50.350356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:50.351537 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:50.351833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:50.352136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] PATH_INFO: `/v3/users/8b8e9d78546d4fd7a3b3090c4aff31c3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:50.353412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b8e9d78546d4fd7a3b3090c4aff31c3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:50.355438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:50.438638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b8e9d78546d4fd7a3b3090c4aff31c3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.439370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b8e9d78546d4fd7a3b3090c4aff31c3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.448299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b8e9d78546d4fd7a3b3090c4aff31c3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.463205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b8e9d78546d4fd7a3b3090c4aff31c3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.473761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fd3d5310-dd92-4de3-835a-5047432a4565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b8e9d78546d4fd7a3b3090c4aff31c3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.481732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1124/2239] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:19:50 2022] DELETE /identity/v3/users/8b8e9d78546d4fd7a3b3090c4aff31c3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:19:50.795069 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:50.798975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:50.816043 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d039ba-21d2-45fa-90d8-19549efb16a1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:50.877987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:50.878912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:50.879178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:50.879525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] PATH_INFO: `/v3/projects/d448706e6bff4b93b746096574062eb6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:50.881027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d448706e6bff4b93b746096574062eb6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:50.901135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:50.985290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd448706e6bff4b93b746096574062eb6', 'request_id': 'req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.992403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd448706e6bff4b93b746096574062eb6', 'request_id': 'req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:50.998447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd448706e6bff4b93b746096574062eb6', 'request_id': 'req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:51.002526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:19:51.015828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] The token cache is being invalidate because project d448706e6bff4b93b746096574062eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:19:51.016580 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:51.019100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:19:51.020450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1116/2240] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:19:50 2022] DELETE /identity/v3/projects/d448706e6bff4b93b746096574062eb6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:19:55.078873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e630-e116-42c2-91dd-f4e59be72c9c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:55.079468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e630-e116-42c2-91dd-f4e59be72c9c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:55.079701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e630-e116-42c2-91dd-f4e59be72c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:55.102122 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-07b8e630-e116-42c2-91dd-f4e59be72c9c None None] MFA Rules not processed for user `99798aca08904bc787f5d0afee8d0ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:19:55.103813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-07b8e630-e116-42c2-91dd-f4e59be72c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:55.146944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:19:55.180272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1125/2241] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:19:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:19:55.557255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:55.559433 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:55.638083 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253cb26-e7ab-4eaa-945b-d29ec7ad3a8f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:55.641872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ba91f-3a4f-4099-9dce-0efcde7e99b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:55.642642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ba91f-3a4f-4099-9dce-0efcde7e99b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:55.642991 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ba91f-3a4f-4099-9dce-0efcde7e99b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:55.643371 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ba91f-3a4f-4099-9dce-0efcde7e99b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:55.645552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8d9ba91f-3a4f-4099-9dce-0efcde7e99b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:55.661399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ba91f-3a4f-4099-9dce-0efcde7e99b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:55.664355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ba91f-3a4f-4099-9dce-0efcde7e99b3 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:55.675883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1117/2242] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:19:57.027661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776c7cf-007b-40be-92f9-c044afe7df30 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:57.028972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776c7cf-007b-40be-92f9-c044afe7df30 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:57.029261 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776c7cf-007b-40be-92f9-c044afe7df30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:57.058138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5776c7cf-007b-40be-92f9-c044afe7df30 None None] MFA Rules not processed for user `691877dce1024aeda61629073d74602b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:19:57.059634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5776c7cf-007b-40be-92f9-c044afe7df30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:57.154042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1126/2243] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:19:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:19:59.099711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ba91f-3a4f-4099-9dce-0efcde7e99b3 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:19:59.109578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ba91f-3a4f-4099-9dce-0efcde7e99b3 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:19:59.114128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3b096-5d89-42c1-b3cf-016f2d382140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:19:59.116197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3b096-5d89-42c1-b3cf-016f2d382140 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:19:59.116687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3b096-5d89-42c1-b3cf-016f2d382140 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:19:59.117225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3b096-5d89-42c1-b3cf-016f2d382140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:19:59.120450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-adb3b096-5d89-42c1-b3cf-016f2d382140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:19:59.229227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3b096-5d89-42c1-b3cf-016f2d382140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:19:59.231231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3b096-5d89-42c1-b3cf-016f2d382140 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:19:59.243445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1118/2244] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:01.934710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776c7cf-007b-40be-92f9-c044afe7df30 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:01.936824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5776c7cf-007b-40be-92f9-c044afe7df30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:02.069799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776c7cf-007b-40be-92f9-c044afe7df30 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:02.088189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfad048-117f-41dc-a498-b6d5b02d4195 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:02.088986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfad048-117f-41dc-a498-b6d5b02d4195 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:02.089385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfad048-117f-41dc-a498-b6d5b02d4195 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:02.089774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfad048-117f-41dc-a498-b6d5b02d4195 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:02.092000 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2bfad048-117f-41dc-a498-b6d5b02d4195 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:02.118149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfad048-117f-41dc-a498-b6d5b02d4195 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:02.121184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfad048-117f-41dc-a498-b6d5b02d4195 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:02.139789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1127/2245] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:05.231079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b9e3c-e8d4-4b51-a375-197aa2b4ca03 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:05.231717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b9e3c-e8d4-4b51-a375-197aa2b4ca03 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:05.231966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b9e3c-e8d4-4b51-a375-197aa2b4ca03 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:05.233556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1119/2246] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:20:05.279160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b2132-8d36-4ecd-b4cb-74bedb4e58ed None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:05.280845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b2132-8d36-4ecd-b4cb-74bedb4e58ed None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:05.281309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b2132-8d36-4ecd-b4cb-74bedb4e58ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:05.305597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f06b2132-8d36-4ecd-b4cb-74bedb4e58ed None None] MFA Rules not processed for user `88ecd4c645174d28930a44b93cfee9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:05.306996 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f06b2132-8d36-4ecd-b4cb-74bedb4e58ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:05.399719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b9e3c-e8d4-4b51-a375-197aa2b4ca03 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:05.403109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-790b9e3c-e8d4-4b51-a375-197aa2b4ca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:05.419098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1128/2247] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:20:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:05.443738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b2132-8d36-4ecd-b4cb-74bedb4e58ed None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:05.445932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f06b2132-8d36-4ecd-b4cb-74bedb4e58ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:05.452177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b9e3c-e8d4-4b51-a375-197aa2b4ca03 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:05.457464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:05.459237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:05.459739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:05.460177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] PATH_INFO: `/v3/users/7c2c0270618240de8fd9290d01a7413a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:05.461742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2c0270618240de8fd9290d01a7413a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:05.464138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:05.474781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b2132-8d36-4ecd-b4cb-74bedb4e58ed None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:05.485486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac430d2-6ec2-432c-9fae-ba549a6790d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:05.486825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac430d2-6ec2-432c-9fae-ba549a6790d6 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:05.491072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac430d2-6ec2-432c-9fae-ba549a6790d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:05.491687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac430d2-6ec2-432c-9fae-ba549a6790d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:05.500234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:05.509276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac430d2-6ec2-432c-9fae-ba549a6790d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:05.512105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac430d2-6ec2-432c-9fae-ba549a6790d6 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:05.532920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1129/2248] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:20:05.561625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2c0270618240de8fd9290d01a7413a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:05.571339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2c0270618240de8fd9290d01a7413a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:05.579795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2c0270618240de8fd9290d01a7413a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:05.601987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2c0270618240de8fd9290d01a7413a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:05.603048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4864aa72-24d7-461f-b693-3e961d572775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2c0270618240de8fd9290d01a7413a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:05.615371 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1120/2249] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:20:05 2022] DELETE /identity/v3/users/7c2c0270618240de8fd9290d01a7413a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:05.845997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac430d2-6ec2-432c-9fae-ba549a6790d6 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:05.862059 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac430d2-6ec2-432c-9fae-ba549a6790d6 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:05.925125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:05.926008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:05.926302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:05.926585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] PATH_INFO: `/v3/projects/601d12851417478a8b39bbf0c9fcd03d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:05.928515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=601d12851417478a8b39bbf0c9fcd03d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:05.930437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:05.985006 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '601d12851417478a8b39bbf0c9fcd03d', 'request_id': 'req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:05.994340 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '601d12851417478a8b39bbf0c9fcd03d', 'request_id': 'req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:06.001072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '601d12851417478a8b39bbf0c9fcd03d', 'request_id': 'req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:06.015657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] The token cache is being invalidate because project 601d12851417478a8b39bbf0c9fcd03d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:20:06.016939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:06.017796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-65bf9d7a-7b97-47dd-a282-7a4e10a26d88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:06.020253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1130/2250] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:20:05 2022] DELETE /identity/v3/projects/601d12851417478a8b39bbf0c9fcd03d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:06.026227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9c4b-cffa-4dfc-973a-3d3098e02bd1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.026889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9c4b-cffa-4dfc-973a-3d3098e02bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.027544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9c4b-cffa-4dfc-973a-3d3098e02bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.044229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:06.070394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-811f9c4b-cffa-4dfc-973a-3d3098e02bd1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:06.071799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-811f9c4b-cffa-4dfc-973a-3d3098e02bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:06.154545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1121/2251] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:06.158481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244380c-11a7-4c57-a15f-2d94bbba646c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.159279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244380c-11a7-4c57-a15f-2d94bbba646c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.159803 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244380c-11a7-4c57-a15f-2d94bbba646c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.190665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7244380c-11a7-4c57-a15f-2d94bbba646c None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:06.191939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7244380c-11a7-4c57-a15f-2d94bbba646c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:06.200582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1131/2252] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:06.205054 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f9c4b-cffa-4dfc-973a-3d3098e02bd1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:06.215808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f9c4b-cffa-4dfc-973a-3d3098e02bd1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:06.220940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d138-2682-4612-9075-f5540383cec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:06.221621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d138-2682-4612-9075-f5540383cec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.221912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d138-2682-4612-9075-f5540383cec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.222194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d138-2682-4612-9075-f5540383cec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.222937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d138-2682-4612-9075-f5540383cec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:20:06.223278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d138-2682-4612-9075-f5540383cec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:06.226299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d138-2682-4612-9075-f5540383cec1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:06.243745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1122/2253] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:20:06.245889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244380c-11a7-4c57-a15f-2d94bbba646c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:06.257118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244380c-11a7-4c57-a15f-2d94bbba646c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:06.263713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e88b6-9597-4913-a5c6-4ec743196003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:06.264406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e88b6-9597-4913-a5c6-4ec743196003 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.264702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e88b6-9597-4913-a5c6-4ec743196003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.264999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e88b6-9597-4913-a5c6-4ec743196003 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.265771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e88b6-9597-4913-a5c6-4ec743196003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:06.268161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e88b6-9597-4913-a5c6-4ec743196003 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:06.306181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1132/2254] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:20:06 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:06.307663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d138-2682-4612-9075-f5540383cec1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:06.320115 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d138-2682-4612-9075-f5540383cec1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:06.390395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca9b13-4de3-4f58-9dc2-240f70bc4bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:06.391093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca9b13-4de3-4f58-9dc2-240f70bc4bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.392968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca9b13-4de3-4f58-9dc2-240f70bc4bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.392968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca9b13-4de3-4f58-9dc2-240f70bc4bfa admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.392968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca9b13-4de3-4f58-9dc2-240f70bc4bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:06.394362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca9b13-4de3-4f58-9dc2-240f70bc4bfa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:06.424510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1123/2255] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:20:06 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:06.430880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e88b6-9597-4913-a5c6-4ec743196003 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:06.443216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e88b6-9597-4913-a5c6-4ec743196003 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:06.448119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc5743-3a20-4b39-b958-fdbfa6ee15c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:06.448781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc5743-3a20-4b39-b958-fdbfa6ee15c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.449077 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc5743-3a20-4b39-b958-fdbfa6ee15c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.449359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc5743-3a20-4b39-b958-fdbfa6ee15c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.450095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc5743-3a20-4b39-b958-fdbfa6ee15c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:06.451745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc5743-3a20-4b39-b958-fdbfa6ee15c8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:06.465558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1133/2256] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:06.472197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca9b13-4de3-4f58-9dc2-240f70bc4bfa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:06.485757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca9b13-4de3-4f58-9dc2-240f70bc4bfa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:06.491023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13572df-8c0c-42f9-9a19-25795f6193f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:06.491724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13572df-8c0c-42f9-9a19-25795f6193f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.492020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13572df-8c0c-42f9-9a19-25795f6193f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.492432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13572df-8c0c-42f9-9a19-25795f6193f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.493184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13572df-8c0c-42f9-9a19-25795f6193f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:06.494798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13572df-8c0c-42f9-9a19-25795f6193f2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:06.506943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1124/2257] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:06.511377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc5743-3a20-4b39-b958-fdbfa6ee15c8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:06.537709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc5743-3a20-4b39-b958-fdbfa6ee15c8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:06.542590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66746ea7-ea15-4ba2-bcfa-3d3c62e2cd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:06.543561 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66746ea7-ea15-4ba2-bcfa-3d3c62e2cd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.543851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66746ea7-ea15-4ba2-bcfa-3d3c62e2cd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.544135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66746ea7-ea15-4ba2-bcfa-3d3c62e2cd5a admin admin] PATH_INFO: `/v3/projects/daa1d1e6c05f4bc7a7a60833876a41a6/users/2e9438e03bc84bea90a3635b5533b325/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.570651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66746ea7-ea15-4ba2-bcfa-3d3c62e2cd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa1d1e6c05f4bc7a7a60833876a41a6, user_id=2e9438e03bc84bea90a3635b5533b325, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:06.573415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66746ea7-ea15-4ba2-bcfa-3d3c62e2cd5a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:06.604839 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1134/2258] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:06 2022] PUT /identity/v3/projects/daa1d1e6c05f4bc7a7a60833876a41a6/users/2e9438e03bc84bea90a3635b5533b325/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:06.615637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4c325-47b2-458f-a360-afbf5e8614e4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.615934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4c325-47b2-458f-a360-afbf5e8614e4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.616219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4c325-47b2-458f-a360-afbf5e8614e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.659240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-15f4c325-47b2-458f-a360-afbf5e8614e4 None None] MFA Rules not processed for user `2e9438e03bc84bea90a3635b5533b325`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:06.660531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-15f4c325-47b2-458f-a360-afbf5e8614e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:06.761706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:06.770860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1125/2259] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:06.772129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66746ea7-ea15-4ba2-bcfa-3d3c62e2cd5a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:06.804047 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:06.857626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66746ea7-ea15-4ba2-bcfa-3d3c62e2cd5a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:06.863041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360254a0-f1e6-45f7-8485-0aaff73ffcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:06.864041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360254a0-f1e6-45f7-8485-0aaff73ffcc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:06.865112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360254a0-f1e6-45f7-8485-0aaff73ffcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:06.865454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360254a0-f1e6-45f7-8485-0aaff73ffcc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:06.866606 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360254a0-f1e6-45f7-8485-0aaff73ffcc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:06.869076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360254a0-f1e6-45f7-8485-0aaff73ffcc8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:06.908128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1135/2260] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:20:06 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:06.909744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4c325-47b2-458f-a360-afbf5e8614e4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:06.924000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4c325-47b2-458f-a360-afbf5e8614e4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:07.018485 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612b783-92d9-4265-a0d5-6b07bc22ade3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:07.019399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612b783-92d9-4265-a0d5-6b07bc22ade3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:07.019903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612b783-92d9-4265-a0d5-6b07bc22ade3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:07.020388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612b783-92d9-4265-a0d5-6b07bc22ade3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:07.021888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612b783-92d9-4265-a0d5-6b07bc22ade3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:07.024339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612b783-92d9-4265-a0d5-6b07bc22ade3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:07.048066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1126/2261] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:20:06 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:07.053202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360254a0-f1e6-45f7-8485-0aaff73ffcc8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:07.065965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360254a0-f1e6-45f7-8485-0aaff73ffcc8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:07.074510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fb32f-0156-46f3-8d3f-e1427d59e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:07.075571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fb32f-0156-46f3-8d3f-e1427d59e1ef admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:07.075944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fb32f-0156-46f3-8d3f-e1427d59e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:07.076300 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fb32f-0156-46f3-8d3f-e1427d59e1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:07.077158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fb32f-0156-46f3-8d3f-e1427d59e1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:07.079026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fb32f-0156-46f3-8d3f-e1427d59e1ef admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:07.093702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1136/2262] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:07.098206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612b783-92d9-4265-a0d5-6b07bc22ade3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:07.109628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612b783-92d9-4265-a0d5-6b07bc22ade3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:07.116250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929be4d3-9e00-421e-b2a1-892c58819343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:07.117944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929be4d3-9e00-421e-b2a1-892c58819343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:07.118555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929be4d3-9e00-421e-b2a1-892c58819343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:07.119051 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929be4d3-9e00-421e-b2a1-892c58819343 admin admin] PATH_INFO: `/v3/projects/37ad6ce0ce314110863fdffef5056977/users/b7553b4ea5514cc0863cb95feedb846f/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:07.142182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929be4d3-9e00-421e-b2a1-892c58819343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ad6ce0ce314110863fdffef5056977, user_id=b7553b4ea5514cc0863cb95feedb846f, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:07.145093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929be4d3-9e00-421e-b2a1-892c58819343 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:07.173874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1127/2263] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:07 2022] PUT /identity/v3/projects/37ad6ce0ce314110863fdffef5056977/users/b7553b4ea5514cc0863cb95feedb846f/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:07.178187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fb32f-0156-46f3-8d3f-e1427d59e1ef admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:07.188668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fb32f-0156-46f3-8d3f-e1427d59e1ef admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:07.257773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c34e2-8ee0-4c63-957d-3b531bb20263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:07.259039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c34e2-8ee0-4c63-957d-3b531bb20263 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:07.259644 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c34e2-8ee0-4c63-957d-3b531bb20263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:07.260076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c34e2-8ee0-4c63-957d-3b531bb20263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:07.261105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c34e2-8ee0-4c63-957d-3b531bb20263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:07.263205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c34e2-8ee0-4c63-957d-3b531bb20263 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:07.276470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1137/2264] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:07.281775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929be4d3-9e00-421e-b2a1-892c58819343 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:07.302809 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929be4d3-9e00-421e-b2a1-892c58819343 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:07.310740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fc867-94a1-42a0-9803-1702e946e676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:07.312042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fc867-94a1-42a0-9803-1702e946e676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:07.312186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fc867-94a1-42a0-9803-1702e946e676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:07.312423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fc867-94a1-42a0-9803-1702e946e676 admin admin] PATH_INFO: `/v3/projects/37ad6ce0ce314110863fdffef5056977/users/b7553b4ea5514cc0863cb95feedb846f/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:07.315034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fc867-94a1-42a0-9803-1702e946e676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ad6ce0ce314110863fdffef5056977, user_id=b7553b4ea5514cc0863cb95feedb846f, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:07.317907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fc867-94a1-42a0-9803-1702e946e676 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:07.330642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1128/2265] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:07 2022] PUT /identity/v3/projects/37ad6ce0ce314110863fdffef5056977/users/b7553b4ea5514cc0863cb95feedb846f/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:07.341364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd2048-1498-444b-ab0c-8a0b9875bd2f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:07.341898 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd2048-1498-444b-ab0c-8a0b9875bd2f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:07.342012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd2048-1498-444b-ab0c-8a0b9875bd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:07.371920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-86dd2048-1498-444b-ab0c-8a0b9875bd2f None None] MFA Rules not processed for user `b7553b4ea5514cc0863cb95feedb846f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:07.373346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-86dd2048-1498-444b-ab0c-8a0b9875bd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:07.485244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1138/2266] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:07.503639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fc867-94a1-42a0-9803-1702e946e676 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:07.506671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a41fc867-94a1-42a0-9803-1702e946e676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:07.586447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fc867-94a1-42a0-9803-1702e946e676 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:07.591876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612e1c-6b9a-4886-8c41-36fd5fecf842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:07.592568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612e1c-6b9a-4886-8c41-36fd5fecf842 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:07.592858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612e1c-6b9a-4886-8c41-36fd5fecf842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:07.593145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612e1c-6b9a-4886-8c41-36fd5fecf842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:07.615722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a612e1c-6b9a-4886-8c41-36fd5fecf842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:07.618557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a612e1c-6b9a-4886-8c41-36fd5fecf842 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:07.639703 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1129/2267] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:20:08.237837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd2048-1498-444b-ab0c-8a0b9875bd2f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:08.279432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:08.381342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd2048-1498-444b-ab0c-8a0b9875bd2f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:08.392449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:08.393659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:08.394405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:08.395152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] PATH_INFO: `/v3/users/2e9438e03bc84bea90a3635b5533b325` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:08.397637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e9438e03bc84bea90a3635b5533b325)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:08.401516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:08.507143 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e9438e03bc84bea90a3635b5533b325'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:08.508269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e9438e03bc84bea90a3635b5533b325'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:08.535857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e9438e03bc84bea90a3635b5533b325'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:08.555851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e9438e03bc84bea90a3635b5533b325'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:08.565694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e9438e03bc84bea90a3635b5533b325'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:08.574552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1139/2268] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:20:08 2022] DELETE /identity/v3/users/2e9438e03bc84bea90a3635b5533b325 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:08.580457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1929f0a-173c-4dd3-9ea8-d79c2963b276 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:08.580959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1929f0a-173c-4dd3-9ea8-d79c2963b276 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:08.581056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1929f0a-173c-4dd3-9ea8-d79c2963b276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:08.623046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c1929f0a-173c-4dd3-9ea8-d79c2963b276 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:08.623046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1929f0a-173c-4dd3-9ea8-d79c2963b276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:08.740952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:08.765242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1130/2269] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:08.772414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:08.774322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:08.857366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be574d74-044e-4f39-b0ce-73df77454831 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:08.861774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78075989-3643-400d-ab95-0c8ad35adf17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:08.862540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78075989-3643-400d-ab95-0c8ad35adf17 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:08.862937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78075989-3643-400d-ab95-0c8ad35adf17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:08.863357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78075989-3643-400d-ab95-0c8ad35adf17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:08.880621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78075989-3643-400d-ab95-0c8ad35adf17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:08.882348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78075989-3643-400d-ab95-0c8ad35adf17 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:08.893464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1140/2270] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:20:09.646542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1929f0a-173c-4dd3-9ea8-d79c2963b276 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:09.648119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1929f0a-173c-4dd3-9ea8-d79c2963b276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:09.667632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1929f0a-173c-4dd3-9ea8-d79c2963b276 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:09.687430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:09.689004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:09.689369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:09.689720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] PATH_INFO: `/v3/projects/daa1d1e6c05f4bc7a7a60833876a41a6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:09.691718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa1d1e6c05f4bc7a7a60833876a41a6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:09.696443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:09.783290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa1d1e6c05f4bc7a7a60833876a41a6', 'request_id': 'req-ffd28a23-6c88-48ba-9def-b13b8ed41362'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:09.796843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa1d1e6c05f4bc7a7a60833876a41a6', 'request_id': 'req-ffd28a23-6c88-48ba-9def-b13b8ed41362'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:09.808962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa1d1e6c05f4bc7a7a60833876a41a6', 'request_id': 'req-ffd28a23-6c88-48ba-9def-b13b8ed41362'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:09.827844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] The token cache is being invalidate because project daa1d1e6c05f4bc7a7a60833876a41a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:20:09.828421 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:09.829230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:09.834589 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1131/2271] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:20:09 2022] DELETE /identity/v3/projects/daa1d1e6c05f4bc7a7a60833876a41a6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:20:09.837051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78075989-3643-400d-ab95-0c8ad35adf17 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:09.870656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78075989-3643-400d-ab95-0c8ad35adf17 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:09.914113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:09.959431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:09.961103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:09.961613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:09.962098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] PATH_INFO: `/v3/users/b7553b4ea5514cc0863cb95feedb846f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:09.964238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7553b4ea5514cc0863cb95feedb846f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:09.970823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:10.052447 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7553b4ea5514cc0863cb95feedb846f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.053280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7553b4ea5514cc0863cb95feedb846f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.068076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7553b4ea5514cc0863cb95feedb846f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.084247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7553b4ea5514cc0863cb95feedb846f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.097242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-913112c7-90ed-40e7-8066-d726276785af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7553b4ea5514cc0863cb95feedb846f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.104148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1141/2272] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:20:09 2022] DELETE /identity/v3/users/b7553b4ea5514cc0863cb95feedb846f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:10.545555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:10.547024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:10.574981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd28a23-6c88-48ba-9def-b13b8ed41362 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:10.678233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:10.679827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:10.680278 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:10.680774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] PATH_INFO: `/v3/projects/37ad6ce0ce314110863fdffef5056977` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:10.682153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ad6ce0ce314110863fdffef5056977)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:10.684434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:10.743073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ad6ce0ce314110863fdffef5056977', 'request_id': 'req-f6568072-6668-424d-952c-eca4700e6440'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.749918 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ad6ce0ce314110863fdffef5056977', 'request_id': 'req-f6568072-6668-424d-952c-eca4700e6440'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.757912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ad6ce0ce314110863fdffef5056977', 'request_id': 'req-f6568072-6668-424d-952c-eca4700e6440'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.774327 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] The token cache is being invalidate because project 37ad6ce0ce314110863fdffef5056977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:20:10.774620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.775320 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f6568072-6668-424d-952c-eca4700e6440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:10.776901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1132/2273] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:20:10 2022] DELETE /identity/v3/projects/37ad6ce0ce314110863fdffef5056977 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:10.782702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fba72d-54c9-47fa-8afe-07cecdcab930 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:10.783104 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fba72d-54c9-47fa-8afe-07cecdcab930 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:10.783548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fba72d-54c9-47fa-8afe-07cecdcab930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:10.805173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-87fba72d-54c9-47fa-8afe-07cecdcab930 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:10.806508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-87fba72d-54c9-47fa-8afe-07cecdcab930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:10.881374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1142/2274] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:20:10.896086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c65ad-d988-4764-9b87-e6c4d9a588a2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:10.896432 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c65ad-d988-4764-9b87-e6c4d9a588a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:10.896750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c65ad-d988-4764-9b87-e6c4d9a588a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:10.920579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4a7c65ad-d988-4764-9b87-e6c4d9a588a2 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:10.921907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4a7c65ad-d988-4764-9b87-e6c4d9a588a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:10.930971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1133/2275] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:10.934937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fba72d-54c9-47fa-8afe-07cecdcab930 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:10.946594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fba72d-54c9-47fa-8afe-07cecdcab930 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:10.951914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b584-235d-4ec4-a902-e879b141cfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:10.952591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b584-235d-4ec4-a902-e879b141cfbc admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:10.952880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b584-235d-4ec4-a902-e879b141cfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:10.953159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b584-235d-4ec4-a902-e879b141cfbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:10.953902 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b584-235d-4ec4-a902-e879b141cfbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:20:10.954209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b584-235d-4ec4-a902-e879b141cfbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:10.955919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b584-235d-4ec4-a902-e879b141cfbc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:10.974020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1143/2276] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:20:10.975286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c65ad-d988-4764-9b87-e6c4d9a588a2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:10.979698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:10.990378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c65ad-d988-4764-9b87-e6c4d9a588a2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:10.998039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d0fbb-3069-4e52-bffb-2c625919845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:10.998852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d0fbb-3069-4e52-bffb-2c625919845b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:10.999243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d0fbb-3069-4e52-bffb-2c625919845b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:10.999660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d0fbb-3069-4e52-bffb-2c625919845b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.000457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d0fbb-3069-4e52-bffb-2c625919845b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:11.002494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d0fbb-3069-4e52-bffb-2c625919845b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:11.043155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1134/2277] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:20:10 2022] POST /identity/v3/projects => generated 363 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:20:11.044594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b584-235d-4ec4-a902-e879b141cfbc admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:11.061696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b584-235d-4ec4-a902-e879b141cfbc admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:11.133381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:11.162645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247808ce-895d-45a9-8bce-6b66dc208537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:11.163799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247808ce-895d-45a9-8bce-6b66dc208537 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:11.164286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247808ce-895d-45a9-8bce-6b66dc208537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:11.164747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247808ce-895d-45a9-8bce-6b66dc208537 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.165678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247808ce-895d-45a9-8bce-6b66dc208537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:11.168127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247808ce-895d-45a9-8bce-6b66dc208537 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:11.204201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1144/2278] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:20:11 2022] POST /identity/v3/users => generated 282 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:11.208126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d0fbb-3069-4e52-bffb-2c625919845b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:11.221307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d0fbb-3069-4e52-bffb-2c625919845b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:11.227822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aac60-b3a0-4c49-8200-b26cbde6adc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:11.228964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aac60-b3a0-4c49-8200-b26cbde6adc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:11.229424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aac60-b3a0-4c49-8200-b26cbde6adc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:11.229931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aac60-b3a0-4c49-8200-b26cbde6adc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.230869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8aac60-b3a0-4c49-8200-b26cbde6adc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:11.232573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8aac60-b3a0-4c49-8200-b26cbde6adc0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:11.263455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1135/2279] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:20:11.267035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247808ce-895d-45a9-8bce-6b66dc208537 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:11.281052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247808ce-895d-45a9-8bce-6b66dc208537 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:11.288609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3079d87-a8fd-4482-ba3c-233dcd9f24d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:11.289849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3079d87-a8fd-4482-ba3c-233dcd9f24d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:11.290417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3079d87-a8fd-4482-ba3c-233dcd9f24d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:11.291039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3079d87-a8fd-4482-ba3c-233dcd9f24d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.292783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3079d87-a8fd-4482-ba3c-233dcd9f24d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:11.296452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3079d87-a8fd-4482-ba3c-233dcd9f24d3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:11.312587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1145/2280] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:11.319422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aac60-b3a0-4c49-8200-b26cbde6adc0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:11.330932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aac60-b3a0-4c49-8200-b26cbde6adc0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:11.336778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1561875-2546-4913-bc42-1cd345c29605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:11.338314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1561875-2546-4913-bc42-1cd345c29605 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:11.339581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1561875-2546-4913-bc42-1cd345c29605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:11.339581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1561875-2546-4913-bc42-1cd345c29605 admin admin] PATH_INFO: `/v3/projects/832bf98926b24bbd9d9760f5a9258d33/users/e62ea67fcab74f40b485c64081861a22/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.361189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1561875-2546-4913-bc42-1cd345c29605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=832bf98926b24bbd9d9760f5a9258d33, user_id=e62ea67fcab74f40b485c64081861a22, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:11.365360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1561875-2546-4913-bc42-1cd345c29605 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:11.385627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1136/2281] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:11 2022] PUT /identity/v3/projects/832bf98926b24bbd9d9760f5a9258d33/users/e62ea67fcab74f40b485c64081861a22/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:11.396062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf87dc-1972-402b-a09f-d519821b5f8d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:11.396355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf87dc-1972-402b-a09f-d519821b5f8d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:11.396639 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf87dc-1972-402b-a09f-d519821b5f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.437015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-71cf87dc-1972-402b-a09f-d519821b5f8d None None] MFA Rules not processed for user `e62ea67fcab74f40b485c64081861a22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:11.438343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-71cf87dc-1972-402b-a09f-d519821b5f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:11.529706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1146/2282] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:11 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:11.537305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1561875-2546-4913-bc42-1cd345c29605 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:11.651581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1561875-2546-4913-bc42-1cd345c29605 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:11.657656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781d327-ff63-4bd5-9bd3-3b61ec2cd910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:11.658567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781d327-ff63-4bd5-9bd3-3b61ec2cd910 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:11.658964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781d327-ff63-4bd5-9bd3-3b61ec2cd910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:11.659385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781d327-ff63-4bd5-9bd3-3b61ec2cd910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.660197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781d327-ff63-4bd5-9bd3-3b61ec2cd910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:11.662814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781d327-ff63-4bd5-9bd3-3b61ec2cd910 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:11.690726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1137/2283] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:20:11 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:11.697741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf87dc-1972-402b-a09f-d519821b5f8d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:11.710912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf87dc-1972-402b-a09f-d519821b5f8d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:11.805102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38a5d2-8109-4082-ad0b-38b451c22221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:11.805912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38a5d2-8109-4082-ad0b-38b451c22221 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:11.806303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38a5d2-8109-4082-ad0b-38b451c22221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:11.806685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38a5d2-8109-4082-ad0b-38b451c22221 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.807768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38a5d2-8109-4082-ad0b-38b451c22221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:11.810077 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38a5d2-8109-4082-ad0b-38b451c22221 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:11.835661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1147/2284] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:20:11 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:11.840442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781d327-ff63-4bd5-9bd3-3b61ec2cd910 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:11.851574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781d327-ff63-4bd5-9bd3-3b61ec2cd910 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:11.856499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fd06a-8221-4c79-8f62-1c86dde3ee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:11.857180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fd06a-8221-4c79-8f62-1c86dde3ee66 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:11.857472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fd06a-8221-4c79-8f62-1c86dde3ee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:11.857757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fd06a-8221-4c79-8f62-1c86dde3ee66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.858496 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fd06a-8221-4c79-8f62-1c86dde3ee66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:11.860144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fd06a-8221-4c79-8f62-1c86dde3ee66 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:11.875283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1138/2285] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:20:11.879585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38a5d2-8109-4082-ad0b-38b451c22221 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:11.891843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38a5d2-8109-4082-ad0b-38b451c22221 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:11.900128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6d595-4c99-4033-8775-c69dfa35e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:11.901106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6d595-4c99-4033-8775-c69dfa35e38c admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:11.901461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6d595-4c99-4033-8775-c69dfa35e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:11.901892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6d595-4c99-4033-8775-c69dfa35e38c admin admin] PATH_INFO: `/v3/projects/7a47f1b8643a4f4fb81777bb0b8a0842/users/62b313c123da4789b3ba6c96bd371bbf/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:11.924017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6d595-4c99-4033-8775-c69dfa35e38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a47f1b8643a4f4fb81777bb0b8a0842, user_id=62b313c123da4789b3ba6c96bd371bbf, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:11.926737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6d595-4c99-4033-8775-c69dfa35e38c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:11.941189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1148/2286] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:11 2022] PUT /identity/v3/projects/7a47f1b8643a4f4fb81777bb0b8a0842/users/62b313c123da4789b3ba6c96bd371bbf/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:11.946329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fd06a-8221-4c79-8f62-1c86dde3ee66 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:11.961371 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fd06a-8221-4c79-8f62-1c86dde3ee66 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:12.029713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55515-cab9-4c3a-99db-2816b7ebf943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:12.031096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55515-cab9-4c3a-99db-2816b7ebf943 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:12.031646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55515-cab9-4c3a-99db-2816b7ebf943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:12.032098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55515-cab9-4c3a-99db-2816b7ebf943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:12.033403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55515-cab9-4c3a-99db-2816b7ebf943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:12.036156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55515-cab9-4c3a-99db-2816b7ebf943 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:12.055477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1139/2287] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:12.060488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6d595-4c99-4033-8775-c69dfa35e38c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:12.080379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6d595-4c99-4033-8775-c69dfa35e38c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:12.087989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459fd77-952b-48d7-bc6e-a28ee363214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:12.089798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459fd77-952b-48d7-bc6e-a28ee363214e admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:12.090276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459fd77-952b-48d7-bc6e-a28ee363214e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:12.090767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459fd77-952b-48d7-bc6e-a28ee363214e admin admin] PATH_INFO: `/v3/projects/7a47f1b8643a4f4fb81777bb0b8a0842/users/62b313c123da4789b3ba6c96bd371bbf/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:12.093865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459fd77-952b-48d7-bc6e-a28ee363214e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a47f1b8643a4f4fb81777bb0b8a0842, user_id=62b313c123da4789b3ba6c96bd371bbf, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:12.096560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459fd77-952b-48d7-bc6e-a28ee363214e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:12.109248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1149/2288] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:12 2022] PUT /identity/v3/projects/7a47f1b8643a4f4fb81777bb0b8a0842/users/62b313c123da4789b3ba6c96bd371bbf/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:12.119596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c36ac-e3e3-42d5-aada-588d86866596 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:12.120516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c36ac-e3e3-42d5-aada-588d86866596 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:12.120516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c36ac-e3e3-42d5-aada-588d86866596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:12.166893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f6c36ac-e3e3-42d5-aada-588d86866596 None None] MFA Rules not processed for user `62b313c123da4789b3ba6c96bd371bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:12.168448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f6c36ac-e3e3-42d5-aada-588d86866596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:12.172740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:12.254210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1140/2289] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:12.266980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459fd77-952b-48d7-bc6e-a28ee363214e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:12.268752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2459fd77-952b-48d7-bc6e-a28ee363214e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:12.276305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:12.335434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459fd77-952b-48d7-bc6e-a28ee363214e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:12.338679 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912cb96-eb38-4f89-9131-201173731cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:12.339405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912cb96-eb38-4f89-9131-201173731cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:12.339693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912cb96-eb38-4f89-9131-201173731cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:12.340003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912cb96-eb38-4f89-9131-201173731cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:12.349380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912cb96-eb38-4f89-9131-201173731cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:12.350817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912cb96-eb38-4f89-9131-201173731cb3 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:12.360468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1150/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:12.441238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c36ac-e3e3-42d5-aada-588d86866596 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:12.456909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c36ac-e3e3-42d5-aada-588d86866596 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:12.462668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f8e23-1063-4501-93e2-0d11dad3c264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:12.463756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f8e23-1063-4501-93e2-0d11dad3c264 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:12.464242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f8e23-1063-4501-93e2-0d11dad3c264 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:12.464718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f8e23-1063-4501-93e2-0d11dad3c264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:12.527661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f8e23-1063-4501-93e2-0d11dad3c264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:12.529038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f8e23-1063-4501-93e2-0d11dad3c264 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:12.538114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1141/2291] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:12 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:22.113080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912cb96-eb38-4f89-9131-201173731cb3 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:22.193496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912cb96-eb38-4f89-9131-201173731cb3 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:22.203492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:22.204392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:22.204672 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:22.204953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] PATH_INFO: `/v3/users/e62ea67fcab74f40b485c64081861a22` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:22.206156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e62ea67fcab74f40b485c64081861a22)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:22.208645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:22.257284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e62ea67fcab74f40b485c64081861a22'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.257942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e62ea67fcab74f40b485c64081861a22'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.265457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e62ea67fcab74f40b485c64081861a22'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.278726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e62ea67fcab74f40b485c64081861a22'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.286525 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e62ea67fcab74f40b485c64081861a22'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.295681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1151/2292] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:20:22 2022] DELETE /identity/v3/users/e62ea67fcab74f40b485c64081861a22 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:22.300638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ca52-e2c3-41fe-9d61-5820d9009325 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:22.301193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ca52-e2c3-41fe-9d61-5820d9009325 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:22.301557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ca52-e2c3-41fe-9d61-5820d9009325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:22.324595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-aa63ca52-e2c3-41fe-9d61-5820d9009325 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:22.326839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-aa63ca52-e2c3-41fe-9d61-5820d9009325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:22.389461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1142/2293] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:20:22.399778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:22.401202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:22.436128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:22.475573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cae9f-5e39-4cbe-824a-2b31f9375663 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:22.479303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240804b0-6076-4603-bf73-b68ccfa6dc65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:22.479976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240804b0-6076-4603-bf73-b68ccfa6dc65 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:22.480272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240804b0-6076-4603-bf73-b68ccfa6dc65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:22.480557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240804b0-6076-4603-bf73-b68ccfa6dc65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:22.491744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240804b0-6076-4603-bf73-b68ccfa6dc65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:22.493385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240804b0-6076-4603-bf73-b68ccfa6dc65 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:22.505446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1152/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:22.801050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63ca52-e2c3-41fe-9d61-5820d9009325 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:22.804060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-aa63ca52-e2c3-41fe-9d61-5820d9009325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:22.823779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63ca52-e2c3-41fe-9d61-5820d9009325 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:22.828799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:22.829669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:22.830802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:22.830802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] PATH_INFO: `/v3/projects/832bf98926b24bbd9d9760f5a9258d33` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:22.831468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=832bf98926b24bbd9d9760f5a9258d33)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:22.833563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:22.875087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:22.891711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832bf98926b24bbd9d9760f5a9258d33', 'request_id': 'req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.903067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832bf98926b24bbd9d9760f5a9258d33', 'request_id': 'req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.910870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832bf98926b24bbd9d9760f5a9258d33', 'request_id': 'req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.924603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] The token cache is being invalidate because project 832bf98926b24bbd9d9760f5a9258d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:20:22.925015 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.925928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:22.928767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1143/2295] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:20:22 2022] DELETE /identity/v3/projects/832bf98926b24bbd9d9760f5a9258d33 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:22.933832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240804b0-6076-4603-bf73-b68ccfa6dc65 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:22.944874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240804b0-6076-4603-bf73-b68ccfa6dc65 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:23.001929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:23.002752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:23.003038 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:23.003344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] PATH_INFO: `/v3/users/62b313c123da4789b3ba6c96bd371bbf` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:23.005135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62b313c123da4789b3ba6c96bd371bbf)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:23.007057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:23.055407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62b313c123da4789b3ba6c96bd371bbf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.056071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62b313c123da4789b3ba6c96bd371bbf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.064157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62b313c123da4789b3ba6c96bd371bbf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.078339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62b313c123da4789b3ba6c96bd371bbf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.085669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dd9b26eb-568a-4cea-ad41-908b8a6b5533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62b313c123da4789b3ba6c96bd371bbf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.091961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1153/2296] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:20:22 2022] DELETE /identity/v3/users/62b313c123da4789b3ba6c96bd371bbf => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:20:23.387221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:23.388931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:23.404186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7997d-6cf1-4730-a1b0-b00fd5d299c4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:23.463328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:23.464203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:23.464488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:23.464804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] PATH_INFO: `/v3/projects/7a47f1b8643a4f4fb81777bb0b8a0842` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:23.466071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a47f1b8643a4f4fb81777bb0b8a0842)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:23.468029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:23.524628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a47f1b8643a4f4fb81777bb0b8a0842', 'request_id': 'req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.530598 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a47f1b8643a4f4fb81777bb0b8a0842', 'request_id': 'req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.536559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a47f1b8643a4f4fb81777bb0b8a0842', 'request_id': 'req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.553403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] The token cache is being invalidate because project 7a47f1b8643a4f4fb81777bb0b8a0842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:20:23.553691 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.554346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f60e9bf-ba56-40df-a4cf-7c7c0fb952bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:23.556456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1144/2297] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:20:23 2022] DELETE /identity/v3/projects/7a47f1b8643a4f4fb81777bb0b8a0842 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:23.562340 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99632aa5-ce8d-4438-b7ca-22dab6e46647 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:23.562953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99632aa5-ce8d-4438-b7ca-22dab6e46647 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:23.562953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99632aa5-ce8d-4438-b7ca-22dab6e46647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:23.583518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-99632aa5-ce8d-4438-b7ca-22dab6e46647 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:23.584806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-99632aa5-ce8d-4438-b7ca-22dab6e46647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:23.645334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:23.666171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1154/2298] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:23.673671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fe39f-c5d1-4e39-ab99-0333e2f14a4c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:23.673969 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fe39f-c5d1-4e39-ab99-0333e2f14a4c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:23.674266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fe39f-c5d1-4e39-ab99-0333e2f14a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:23.699761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7a6fe39f-c5d1-4e39-ab99-0333e2f14a4c None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:23.701004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7a6fe39f-c5d1-4e39-ab99-0333e2f14a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:23.716777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1145/2299] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:23.721430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99632aa5-ce8d-4438-b7ca-22dab6e46647 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:23.732725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99632aa5-ce8d-4438-b7ca-22dab6e46647 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:23.737471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e3086-aacb-4773-8741-bef07010c47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:23.738127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3086-aacb-4773-8741-bef07010c47e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:23.738409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3086-aacb-4773-8741-bef07010c47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:23.739177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3086-aacb-4773-8741-bef07010c47e admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:23.740184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e3086-aacb-4773-8741-bef07010c47e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:20:23.740495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e3086-aacb-4773-8741-bef07010c47e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:23.742082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e3086-aacb-4773-8741-bef07010c47e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:23.753863 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1155/2300] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:20:23.758722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fe39f-c5d1-4e39-ab99-0333e2f14a4c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:23.768078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fe39f-c5d1-4e39-ab99-0333e2f14a4c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:23.772896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f5d9c-182f-4baa-b576-6af8cbb3944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:23.773550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f5d9c-182f-4baa-b576-6af8cbb3944b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:23.773835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f5d9c-182f-4baa-b576-6af8cbb3944b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:23.774147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f5d9c-182f-4baa-b576-6af8cbb3944b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:23.774930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f5d9c-182f-4baa-b576-6af8cbb3944b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:23.776800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f5d9c-182f-4baa-b576-6af8cbb3944b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:23.803233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1146/2301] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:20:23 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:20:23.808005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e3086-aacb-4773-8741-bef07010c47e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:23.821040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e3086-aacb-4773-8741-bef07010c47e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:23.901054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321526f-c33e-48be-ba57-68e531aeb58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:23.901735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321526f-c33e-48be-ba57-68e531aeb58b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:23.902021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321526f-c33e-48be-ba57-68e531aeb58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:23.902298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321526f-c33e-48be-ba57-68e531aeb58b admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:23.902992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321526f-c33e-48be-ba57-68e531aeb58b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:23.904943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321526f-c33e-48be-ba57-68e531aeb58b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:23.931087 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1156/2302] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:20:23 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:23.935852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f5d9c-182f-4baa-b576-6af8cbb3944b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:23.949613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f5d9c-182f-4baa-b576-6af8cbb3944b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:23.954630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ab05a-911e-4afb-b0ac-e1ad619e6ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:23.955305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ab05a-911e-4afb-b0ac-e1ad619e6ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:23.955604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ab05a-911e-4afb-b0ac-e1ad619e6ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:23.955880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ab05a-911e-4afb-b0ac-e1ad619e6ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:23.956612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ab05a-911e-4afb-b0ac-e1ad619e6ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:23.958239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ab05a-911e-4afb-b0ac-e1ad619e6ed7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:23.968982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1147/2303] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:20:23.975011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321526f-c33e-48be-ba57-68e531aeb58b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:23.987939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321526f-c33e-48be-ba57-68e531aeb58b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:23.996381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e164cb5-594c-48fd-b127-2f5d5ceaa104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:23.997486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e164cb5-594c-48fd-b127-2f5d5ceaa104 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:23.997972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e164cb5-594c-48fd-b127-2f5d5ceaa104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:23.998415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e164cb5-594c-48fd-b127-2f5d5ceaa104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:23.999626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e164cb5-594c-48fd-b127-2f5d5ceaa104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:24.001682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e164cb5-594c-48fd-b127-2f5d5ceaa104 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:24.013467 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1157/2304] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:20:24.017761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ab05a-911e-4afb-b0ac-e1ad619e6ed7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:24.028935 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ab05a-911e-4afb-b0ac-e1ad619e6ed7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:24.033831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5242b00-0ed8-4b8f-8ebb-ff160ff08ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:24.034717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5242b00-0ed8-4b8f-8ebb-ff160ff08ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:24.035022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5242b00-0ed8-4b8f-8ebb-ff160ff08ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:24.035335 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5242b00-0ed8-4b8f-8ebb-ff160ff08ef0 admin admin] PATH_INFO: `/v3/projects/8c378196e10643fbb8740aba4f82c51b/users/7f9339b2f87f479dba6528ab7c43aa30/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:24.047420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:24.055231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5242b00-0ed8-4b8f-8ebb-ff160ff08ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c378196e10643fbb8740aba4f82c51b, user_id=7f9339b2f87f479dba6528ab7c43aa30, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:24.060168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5242b00-0ed8-4b8f-8ebb-ff160ff08ef0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:24.077864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1148/2305] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:24 2022] PUT /identity/v3/projects/8c378196e10643fbb8740aba4f82c51b/users/7f9339b2f87f479dba6528ab7c43aa30/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:24.088428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5ee7-f538-4b20-9d49-68a9cf1632cc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:24.088721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5ee7-f538-4b20-9d49-68a9cf1632cc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:24.089028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5ee7-f538-4b20-9d49-68a9cf1632cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:24.117651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2f8f5ee7-f538-4b20-9d49-68a9cf1632cc None None] MFA Rules not processed for user `7f9339b2f87f479dba6528ab7c43aa30`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:24.118877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2f8f5ee7-f538-4b20-9d49-68a9cf1632cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:24.201015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1158/2306] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:24.208200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5242b00-0ed8-4b8f-8ebb-ff160ff08ef0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:24.275071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5242b00-0ed8-4b8f-8ebb-ff160ff08ef0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:24.280210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2382f22-e377-4368-9610-6a4e195e8533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:24.280927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2382f22-e377-4368-9610-6a4e195e8533 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:24.281266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2382f22-e377-4368-9610-6a4e195e8533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:24.281596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2382f22-e377-4368-9610-6a4e195e8533 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:24.282417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2382f22-e377-4368-9610-6a4e195e8533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:24.284534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2382f22-e377-4368-9610-6a4e195e8533 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:24.321981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1149/2307] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:20:24 2022] POST /identity/v3/projects => generated 401 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:24.326428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5ee7-f538-4b20-9d49-68a9cf1632cc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:24.339736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5ee7-f538-4b20-9d49-68a9cf1632cc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:24.385154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:24.407775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca7413-464a-41ae-96fb-25cbc69217bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:24.408860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca7413-464a-41ae-96fb-25cbc69217bb admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:24.409372 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca7413-464a-41ae-96fb-25cbc69217bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:24.409870 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca7413-464a-41ae-96fb-25cbc69217bb admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:24.411535 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca7413-464a-41ae-96fb-25cbc69217bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:24.414023 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca7413-464a-41ae-96fb-25cbc69217bb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:24.440959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1159/2308] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:20:24 2022] POST /identity/v3/users => generated 307 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:24.445183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2382f22-e377-4368-9610-6a4e195e8533 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:24.458433 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2382f22-e377-4368-9610-6a4e195e8533 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:24.464795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25108d17-4694-4f08-ae1a-0b6f3ad34423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:24.465853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25108d17-4694-4f08-ae1a-0b6f3ad34423 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:24.466147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25108d17-4694-4f08-ae1a-0b6f3ad34423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:24.466426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25108d17-4694-4f08-ae1a-0b6f3ad34423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:24.467162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25108d17-4694-4f08-ae1a-0b6f3ad34423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:24.468804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25108d17-4694-4f08-ae1a-0b6f3ad34423 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:24.481018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1150/2309] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:20:24.485419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca7413-464a-41ae-96fb-25cbc69217bb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:24.497901 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca7413-464a-41ae-96fb-25cbc69217bb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:24.503328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e0556-0926-41cd-90ed-431fa321e4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:24.504260 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e0556-0926-41cd-90ed-431fa321e4e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:24.504545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e0556-0926-41cd-90ed-431fa321e4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:24.504820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e0556-0926-41cd-90ed-431fa321e4e0 admin admin] PATH_INFO: `/v3/projects/a3171b97a43947fb823cc2e9dfd831e8/users/724a4eb6aca5421fabbe5af6559f18c1/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:24.521618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e0556-0926-41cd-90ed-431fa321e4e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3171b97a43947fb823cc2e9dfd831e8, user_id=724a4eb6aca5421fabbe5af6559f18c1, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:24.524321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e0556-0926-41cd-90ed-431fa321e4e0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:24.538738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1160/2310] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:24 2022] PUT /identity/v3/projects/a3171b97a43947fb823cc2e9dfd831e8/users/724a4eb6aca5421fabbe5af6559f18c1/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:20:24.542863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25108d17-4694-4f08-ae1a-0b6f3ad34423 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:24.552979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25108d17-4694-4f08-ae1a-0b6f3ad34423 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:24.618611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a06f0-3dbb-47ae-908d-ec5a9fae9f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:24.619344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a06f0-3dbb-47ae-908d-ec5a9fae9f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:24.619638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a06f0-3dbb-47ae-908d-ec5a9fae9f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:24.619923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a06f0-3dbb-47ae-908d-ec5a9fae9f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:24.620693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a06f0-3dbb-47ae-908d-ec5a9fae9f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:24.622333 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a06f0-3dbb-47ae-908d-ec5a9fae9f06 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:24.632634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1151/2311] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:24.636989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e0556-0926-41cd-90ed-431fa321e4e0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:24.652582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e0556-0926-41cd-90ed-431fa321e4e0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:24.661697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5a9b7-3884-43cc-87bd-b61e154d894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:24.662876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5a9b7-3884-43cc-87bd-b61e154d894f admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:24.663382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5a9b7-3884-43cc-87bd-b61e154d894f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:24.663810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5a9b7-3884-43cc-87bd-b61e154d894f admin admin] PATH_INFO: `/v3/projects/a3171b97a43947fb823cc2e9dfd831e8/users/724a4eb6aca5421fabbe5af6559f18c1/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:24.665877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5a9b7-3884-43cc-87bd-b61e154d894f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3171b97a43947fb823cc2e9dfd831e8, user_id=724a4eb6aca5421fabbe5af6559f18c1, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:24.668666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5a9b7-3884-43cc-87bd-b61e154d894f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:24.681520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1161/2312] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:24 2022] PUT /identity/v3/projects/a3171b97a43947fb823cc2e9dfd831e8/users/724a4eb6aca5421fabbe5af6559f18c1/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:20:24.698279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cad50a-7237-4817-9c5e-985c234abaab None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:24.698699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cad50a-7237-4817-9c5e-985c234abaab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:24.699047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cad50a-7237-4817-9c5e-985c234abaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:24.727967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-66cad50a-7237-4817-9c5e-985c234abaab None None] MFA Rules not processed for user `724a4eb6aca5421fabbe5af6559f18c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:24.729201 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66cad50a-7237-4817-9c5e-985c234abaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:24.780582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:24.806076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1152/2313] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:24 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:20:24.976065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5a9b7-3884-43cc-87bd-b61e154d894f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:24.981884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-08c5a9b7-3884-43cc-87bd-b61e154d894f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:25.046796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5a9b7-3884-43cc-87bd-b61e154d894f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:25.050797 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372dd661-25df-42b5-bc5c-0e8524d77db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:25.051976 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dd661-25df-42b5-bc5c-0e8524d77db7 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:25.052264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dd661-25df-42b5-bc5c-0e8524d77db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:25.052545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dd661-25df-42b5-bc5c-0e8524d77db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:25.062249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372dd661-25df-42b5-bc5c-0e8524d77db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:25.063694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372dd661-25df-42b5-bc5c-0e8524d77db7 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:25.076334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1162/2314] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:24 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:25.152767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cad50a-7237-4817-9c5e-985c234abaab None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:25.163166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cad50a-7237-4817-9c5e-985c234abaab None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:25.166891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a0a6a-eaed-40eb-9def-19f1d3c57eb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:25.167579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a0a6a-eaed-40eb-9def-19f1d3c57eb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:25.167867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a0a6a-eaed-40eb-9def-19f1d3c57eb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:25.168155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a0a6a-eaed-40eb-9def-19f1d3c57eb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:25.229858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a0a6a-eaed-40eb-9def-19f1d3c57eb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:25.231479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a0a6a-eaed-40eb-9def-19f1d3c57eb4 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:25.241684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1153/2315] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:29.428280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372dd661-25df-42b5-bc5c-0e8524d77db7 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:29.495725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372dd661-25df-42b5-bc5c-0e8524d77db7 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:29.502157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:29.503705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:29.504726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:29.505219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] PATH_INFO: `/v3/users/7f9339b2f87f479dba6528ab7c43aa30` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:29.507359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f9339b2f87f479dba6528ab7c43aa30)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:29.510840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:29.568552 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:29.580001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f9339b2f87f479dba6528ab7c43aa30'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:29.580681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f9339b2f87f479dba6528ab7c43aa30'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:29.588923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f9339b2f87f479dba6528ab7c43aa30'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:29.603616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f9339b2f87f479dba6528ab7c43aa30'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:29.610691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f9339b2f87f479dba6528ab7c43aa30'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:29.617844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1163/2316] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:20:29 2022] DELETE /identity/v3/users/7f9339b2f87f479dba6528ab7c43aa30 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:29.623726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5011b-a7f5-4826-b93b-26a76ea56ec2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:29.624226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5011b-a7f5-4826-b93b-26a76ea56ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:29.624393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5011b-a7f5-4826-b93b-26a76ea56ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:29.644554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8ef5011b-a7f5-4826-b93b-26a76ea56ec2 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:29.645939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8ef5011b-a7f5-4826-b93b-26a76ea56ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:29.724129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1154/2317] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:29.734014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:29.735557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:29.803154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fce41c-b28d-4b6a-9471-d489bec0ceec admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:29.807226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc688a-7b83-4e0a-ac56-5769710c83db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:29.807728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc688a-7b83-4e0a-ac56-5769710c83db service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:29.807993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc688a-7b83-4e0a-ac56-5769710c83db service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:29.808245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc688a-7b83-4e0a-ac56-5769710c83db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:29.820502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc688a-7b83-4e0a-ac56-5769710c83db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:29.821967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc688a-7b83-4e0a-ac56-5769710c83db service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:29.832955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1164/2318] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:20:30.133711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5011b-a7f5-4826-b93b-26a76ea56ec2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:30.135909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8ef5011b-a7f5-4826-b93b-26a76ea56ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:30.158897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5011b-a7f5-4826-b93b-26a76ea56ec2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:30.164907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:30.165787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:30.166075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:30.166359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] PATH_INFO: `/v3/projects/8c378196e10643fbb8740aba4f82c51b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:30.167612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c378196e10643fbb8740aba4f82c51b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:30.169584 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:30.228276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c378196e10643fbb8740aba4f82c51b', 'request_id': 'req-97aeb41f-3176-412d-baae-d61b76dce6d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.234718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c378196e10643fbb8740aba4f82c51b', 'request_id': 'req-97aeb41f-3176-412d-baae-d61b76dce6d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.241982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c378196e10643fbb8740aba4f82c51b', 'request_id': 'req-97aeb41f-3176-412d-baae-d61b76dce6d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.257393 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] The token cache is being invalidate because project 8c378196e10643fbb8740aba4f82c51b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:20:30.257689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.258357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.259973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1155/2319] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:20:30 2022] DELETE /identity/v3/projects/8c378196e10643fbb8740aba4f82c51b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:30.264102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc688a-7b83-4e0a-ac56-5769710c83db service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:30.277424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc688a-7b83-4e0a-ac56-5769710c83db service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:30.347030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:30.348474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:30.348859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:30.349248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] PATH_INFO: `/v3/users/724a4eb6aca5421fabbe5af6559f18c1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:30.350622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724a4eb6aca5421fabbe5af6559f18c1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:30.352768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:30.412688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724a4eb6aca5421fabbe5af6559f18c1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.413356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724a4eb6aca5421fabbe5af6559f18c1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.422150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724a4eb6aca5421fabbe5af6559f18c1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.430348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:30.437038 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724a4eb6aca5421fabbe5af6559f18c1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.445668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2f0c8cb3-74dc-45bb-be16-2bb93f5a0388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724a4eb6aca5421fabbe5af6559f18c1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.451971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1165/2320] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:20:30 2022] DELETE /identity/v3/users/724a4eb6aca5421fabbe5af6559f18c1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:20:30.765479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:30.767738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:30.791365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb41f-3176-412d-baae-d61b76dce6d7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:30.797429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:30.861451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:30.862230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:30.862506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:30.862782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] PATH_INFO: `/v3/projects/a3171b97a43947fb823cc2e9dfd831e8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:30.864032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3171b97a43947fb823cc2e9dfd831e8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:30.866205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:30.922699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3171b97a43947fb823cc2e9dfd831e8', 'request_id': 'req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.930968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3171b97a43947fb823cc2e9dfd831e8', 'request_id': 'req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.937195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3171b97a43947fb823cc2e9dfd831e8', 'request_id': 'req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.951148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] The token cache is being invalidate because project a3171b97a43947fb823cc2e9dfd831e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:20:30.951481 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.952110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-14dce67e-c6c2-451c-9d44-f63ff8f99e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:30.953863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1156/2321] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:20:30 2022] DELETE /identity/v3/projects/a3171b97a43947fb823cc2e9dfd831e8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:30.959750 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e81ea7-67b4-4c79-8598-7cb9e6db8c4b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:30.960043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e81ea7-67b4-4c79-8598-7cb9e6db8c4b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:30.960328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e81ea7-67b4-4c79-8598-7cb9e6db8c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:30.979294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-50e81ea7-67b4-4c79-8598-7cb9e6db8c4b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:30.980632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-50e81ea7-67b4-4c79-8598-7cb9e6db8c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:31.042696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1166/2322] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:31.051489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e627c8-3dfe-42be-bb4a-ddf80931d19d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.051964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e627c8-3dfe-42be-bb4a-ddf80931d19d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.052116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e627c8-3dfe-42be-bb4a-ddf80931d19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.072753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-94e627c8-3dfe-42be-bb4a-ddf80931d19d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:31.074143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-94e627c8-3dfe-42be-bb4a-ddf80931d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:31.082911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1157/2323] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:31.087955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e81ea7-67b4-4c79-8598-7cb9e6db8c4b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.098592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e81ea7-67b4-4c79-8598-7cb9e6db8c4b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.103595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa29f2-8f94-45f8-8ee3-bf60229be509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.104275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa29f2-8f94-45f8-8ee3-bf60229be509 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.104601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa29f2-8f94-45f8-8ee3-bf60229be509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.104888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa29f2-8f94-45f8-8ee3-bf60229be509 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.105627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa29f2-8f94-45f8-8ee3-bf60229be509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:20:31.105937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa29f2-8f94-45f8-8ee3-bf60229be509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.107597 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa29f2-8f94-45f8-8ee3-bf60229be509 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.121636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1167/2324] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:20:31.130040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e627c8-3dfe-42be-bb4a-ddf80931d19d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.139846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e627c8-3dfe-42be-bb4a-ddf80931d19d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.145071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c0ac9-d454-4220-b1db-f2c45d5d7841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.145755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c0ac9-d454-4220-b1db-f2c45d5d7841 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.146050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c0ac9-d454-4220-b1db-f2c45d5d7841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.146332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c0ac9-d454-4220-b1db-f2c45d5d7841 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.147038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c0ac9-d454-4220-b1db-f2c45d5d7841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.149047 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c0ac9-d454-4220-b1db-f2c45d5d7841 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.176694 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1158/2325] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:20:31 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:31.181196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa29f2-8f94-45f8-8ee3-bf60229be509 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.193542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa29f2-8f94-45f8-8ee3-bf60229be509 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.257021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f547a93-c868-4b6d-8742-f2560e6258c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.257701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f547a93-c868-4b6d-8742-f2560e6258c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.257992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f547a93-c868-4b6d-8742-f2560e6258c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.258275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f547a93-c868-4b6d-8742-f2560e6258c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.258974 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f547a93-c868-4b6d-8742-f2560e6258c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.260953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f547a93-c868-4b6d-8742-f2560e6258c7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.285127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1168/2326] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:20:31 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:31.290077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c0ac9-d454-4220-b1db-f2c45d5d7841 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.303188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c0ac9-d454-4220-b1db-f2c45d5d7841 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.308394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e41f5e-650b-47ea-ba47-a3beda1e0726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.309128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e41f5e-650b-47ea-ba47-a3beda1e0726 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.309413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e41f5e-650b-47ea-ba47-a3beda1e0726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.309730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e41f5e-650b-47ea-ba47-a3beda1e0726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.310452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e41f5e-650b-47ea-ba47-a3beda1e0726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.312126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e41f5e-650b-47ea-ba47-a3beda1e0726 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.325224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1159/2327] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:31.339548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f547a93-c868-4b6d-8742-f2560e6258c7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.349894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f547a93-c868-4b6d-8742-f2560e6258c7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.354757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cee2e6-0aea-429d-bef1-6dea049344fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.355468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cee2e6-0aea-429d-bef1-6dea049344fc admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.355758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cee2e6-0aea-429d-bef1-6dea049344fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.356111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cee2e6-0aea-429d-bef1-6dea049344fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.356782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cee2e6-0aea-429d-bef1-6dea049344fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.358396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cee2e6-0aea-429d-bef1-6dea049344fc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.369696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1169/2328] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:31.374213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e41f5e-650b-47ea-ba47-a3beda1e0726 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.385068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e41f5e-650b-47ea-ba47-a3beda1e0726 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.389858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eea1f-fa3a-481a-ac5c-0c5cd4b75074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.390759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eea1f-fa3a-481a-ac5c-0c5cd4b75074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.391042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eea1f-fa3a-481a-ac5c-0c5cd4b75074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.391355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eea1f-fa3a-481a-ac5c-0c5cd4b75074 admin admin] PATH_INFO: `/v3/projects/e96da36bd7a3441f9d671fdc26515e16/users/52f768f9fc3043578f68999d638086b5/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.412958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eea1f-fa3a-481a-ac5c-0c5cd4b75074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96da36bd7a3441f9d671fdc26515e16, user_id=52f768f9fc3043578f68999d638086b5, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.415652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eea1f-fa3a-481a-ac5c-0c5cd4b75074 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.430817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1160/2329] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:31 2022] PUT /identity/v3/projects/e96da36bd7a3441f9d671fdc26515e16/users/52f768f9fc3043578f68999d638086b5/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:31.441293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de189d-5e7a-4bb6-811b-b2a14bb856b3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.441836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de189d-5e7a-4bb6-811b-b2a14bb856b3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.441951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de189d-5e7a-4bb6-811b-b2a14bb856b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.469866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a4de189d-5e7a-4bb6-811b-b2a14bb856b3 None None] MFA Rules not processed for user `52f768f9fc3043578f68999d638086b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:31.471248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a4de189d-5e7a-4bb6-811b-b2a14bb856b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:31.510576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:31.549377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1170/2330] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:20:31.554909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eea1f-fa3a-481a-ac5c-0c5cd4b75074 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.601151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:31.640452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eea1f-fa3a-481a-ac5c-0c5cd4b75074 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.645699 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cb6bd-870d-4ad6-8dd0-c98809d462d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.646390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cb6bd-870d-4ad6-8dd0-c98809d462d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.646681 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cb6bd-870d-4ad6-8dd0-c98809d462d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.646964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cb6bd-870d-4ad6-8dd0-c98809d462d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.647698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cb6bd-870d-4ad6-8dd0-c98809d462d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.649635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cb6bd-870d-4ad6-8dd0-c98809d462d4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.697306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1161/2331] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:20:31 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:31.701675 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de189d-5e7a-4bb6-811b-b2a14bb856b3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.712866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de189d-5e7a-4bb6-811b-b2a14bb856b3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.777247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f1470-b5ee-4f12-81e6-54f13f9933c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.777942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f1470-b5ee-4f12-81e6-54f13f9933c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.778241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f1470-b5ee-4f12-81e6-54f13f9933c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.778506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f1470-b5ee-4f12-81e6-54f13f9933c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.779217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f1470-b5ee-4f12-81e6-54f13f9933c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.781207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f1470-b5ee-4f12-81e6-54f13f9933c6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.813304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1171/2332] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:20:31 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:20:31.818527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cb6bd-870d-4ad6-8dd0-c98809d462d4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.829264 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cb6bd-870d-4ad6-8dd0-c98809d462d4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.834645 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f60be-b400-42b5-b66d-5e7f1552b61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.835319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f60be-b400-42b5-b66d-5e7f1552b61a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.835795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f60be-b400-42b5-b66d-5e7f1552b61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.836048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f60be-b400-42b5-b66d-5e7f1552b61a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.836544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f60be-b400-42b5-b66d-5e7f1552b61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.838180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f60be-b400-42b5-b66d-5e7f1552b61a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.850285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1162/2333] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:20:31.855178 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f1470-b5ee-4f12-81e6-54f13f9933c6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.870200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f1470-b5ee-4f12-81e6-54f13f9933c6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:31.877244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff1cae-f2d0-4b67-a39b-864c6f601fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:31.878146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff1cae-f2d0-4b67-a39b-864c6f601fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:31.878433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff1cae-f2d0-4b67-a39b-864c6f601fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:31.878719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff1cae-f2d0-4b67-a39b-864c6f601fbe admin admin] PATH_INFO: `/v3/projects/c175f4c6bba3472a9f4059b8c7bd0d79/users/529209766f4341a4a9012b0f198e9093/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:31.896242 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff1cae-f2d0-4b67-a39b-864c6f601fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c175f4c6bba3472a9f4059b8c7bd0d79, user_id=529209766f4341a4a9012b0f198e9093, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:31.898856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff1cae-f2d0-4b67-a39b-864c6f601fbe admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:31.918370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1172/2334] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:31 2022] PUT /identity/v3/projects/c175f4c6bba3472a9f4059b8c7bd0d79/users/529209766f4341a4a9012b0f198e9093/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:31.923105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f60be-b400-42b5-b66d-5e7f1552b61a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:31.933982 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f60be-b400-42b5-b66d-5e7f1552b61a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:32.002833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ade4ac-346c-4609-8372-b3920d555f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:32.003537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade4ac-346c-4609-8372-b3920d555f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:32.003823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade4ac-346c-4609-8372-b3920d555f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:32.004100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade4ac-346c-4609-8372-b3920d555f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:32.004828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ade4ac-346c-4609-8372-b3920d555f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:32.006423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ade4ac-346c-4609-8372-b3920d555f2e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:32.016862 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1163/2335] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:32.022536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff1cae-f2d0-4b67-a39b-864c6f601fbe admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:32.034572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff1cae-f2d0-4b67-a39b-864c6f601fbe admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:32.043966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35c4d6-2b40-4e4c-bd8f-7d621ee68b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:32.044893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35c4d6-2b40-4e4c-bd8f-7d621ee68b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:32.045391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35c4d6-2b40-4e4c-bd8f-7d621ee68b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:32.045681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35c4d6-2b40-4e4c-bd8f-7d621ee68b43 admin admin] PATH_INFO: `/v3/projects/c175f4c6bba3472a9f4059b8c7bd0d79/users/529209766f4341a4a9012b0f198e9093/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:32.047835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35c4d6-2b40-4e4c-bd8f-7d621ee68b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c175f4c6bba3472a9f4059b8c7bd0d79, user_id=529209766f4341a4a9012b0f198e9093, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:32.051645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35c4d6-2b40-4e4c-bd8f-7d621ee68b43 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:32.065056 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1173/2336] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:32 2022] PUT /identity/v3/projects/c175f4c6bba3472a9f4059b8c7bd0d79/users/529209766f4341a4a9012b0f198e9093/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:32.077198 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d37372-8333-4503-8ce6-348030ae6be1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:32.077494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d37372-8333-4503-8ce6-348030ae6be1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:32.077816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d37372-8333-4503-8ce6-348030ae6be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:32.109545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-88d37372-8333-4503-8ce6-348030ae6be1 None None] MFA Rules not processed for user `529209766f4341a4a9012b0f198e9093`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:32.110815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-88d37372-8333-4503-8ce6-348030ae6be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:32.187075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1164/2337] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:32.199218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35c4d6-2b40-4e4c-bd8f-7d621ee68b43 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:32.201580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3f35c4d6-2b40-4e4c-bd8f-7d621ee68b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:32.265753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35c4d6-2b40-4e4c-bd8f-7d621ee68b43 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:32.270286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df37db2-7367-40e9-8376-571e65775ae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:32.270951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df37db2-7367-40e9-8376-571e65775ae1 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:32.271244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df37db2-7367-40e9-8376-571e65775ae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:32.271566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df37db2-7367-40e9-8376-571e65775ae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:32.281800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df37db2-7367-40e9-8376-571e65775ae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:32.283296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df37db2-7367-40e9-8376-571e65775ae1 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:32.295845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1174/2338] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:32.415654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d37372-8333-4503-8ce6-348030ae6be1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:32.425801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d37372-8333-4503-8ce6-348030ae6be1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:32.429938 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2a308-076f-42af-8357-5b4bf8ca8e19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:32.430603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2a308-076f-42af-8357-5b4bf8ca8e19 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:32.430951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2a308-076f-42af-8357-5b4bf8ca8e19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:32.431228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2a308-076f-42af-8357-5b4bf8ca8e19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:32.465503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:32.500213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2a308-076f-42af-8357-5b4bf8ca8e19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:32.501745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2a308-076f-42af-8357-5b4bf8ca8e19 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:32.512685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1165/2339] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:35.173746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df37db2-7367-40e9-8376-571e65775ae1 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:35.211543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:35.246682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df37db2-7367-40e9-8376-571e65775ae1 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:35.251502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:35.252398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:35.252680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:35.252960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] PATH_INFO: `/v3/users/52f768f9fc3043578f68999d638086b5` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:35.254196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52f768f9fc3043578f68999d638086b5)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:35.256204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:35.308425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52f768f9fc3043578f68999d638086b5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:35.309234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52f768f9fc3043578f68999d638086b5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:35.328879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52f768f9fc3043578f68999d638086b5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:35.346954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52f768f9fc3043578f68999d638086b5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:35.355053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52f768f9fc3043578f68999d638086b5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:35.362098 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1175/2340] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:20:35 2022] DELETE /identity/v3/users/52f768f9fc3043578f68999d638086b5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:20:35.368200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004cd82-d454-40d7-aea9-08480e34048d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:35.368744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004cd82-d454-40d7-aea9-08480e34048d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:35.368847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004cd82-d454-40d7-aea9-08480e34048d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:35.386704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b004cd82-d454-40d7-aea9-08480e34048d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:35.387983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b004cd82-d454-40d7-aea9-08480e34048d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:35.453401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1166/2341] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:35.463630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:35.465271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:35.539959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b4737-e8e6-425e-92d6-565dfd8aefc5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:35.544530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba9c5c-8740-4c95-946c-b0d9d892c385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:35.545366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba9c5c-8740-4c95-946c-b0d9d892c385 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:35.545794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba9c5c-8740-4c95-946c-b0d9d892c385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:35.546163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba9c5c-8740-4c95-946c-b0d9d892c385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:35.557226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba9c5c-8740-4c95-946c-b0d9d892c385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:35.558915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba9c5c-8740-4c95-946c-b0d9d892c385 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:35.569350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1176/2342] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:35.905310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004cd82-d454-40d7-aea9-08480e34048d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:35.907633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b004cd82-d454-40d7-aea9-08480e34048d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:35.922949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004cd82-d454-40d7-aea9-08480e34048d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:35.930948 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:35.932458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:35.932983 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:35.933498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] PATH_INFO: `/v3/projects/e96da36bd7a3441f9d671fdc26515e16` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:35.935274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96da36bd7a3441f9d671fdc26515e16)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:35.937290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:35.991875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96da36bd7a3441f9d671fdc26515e16', 'request_id': 'req-25bd5deb-8047-4616-ad57-7b992bbd4fc9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:35.999345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96da36bd7a3441f9d671fdc26515e16', 'request_id': 'req-25bd5deb-8047-4616-ad57-7b992bbd4fc9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.007386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96da36bd7a3441f9d671fdc26515e16', 'request_id': 'req-25bd5deb-8047-4616-ad57-7b992bbd4fc9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.023209 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] The token cache is being invalidate because project e96da36bd7a3441f9d671fdc26515e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:20:36.023525 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.024233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.025912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1167/2343] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:20:35 2022] DELETE /identity/v3/projects/e96da36bd7a3441f9d671fdc26515e16 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:20:36.029924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba9c5c-8740-4c95-946c-b0d9d892c385 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:36.040111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba9c5c-8740-4c95-946c-b0d9d892c385 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:36.095964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:36.107463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:36.108394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:36.108681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:36.112344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] PATH_INFO: `/v3/users/529209766f4341a4a9012b0f198e9093` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:36.112344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=529209766f4341a4a9012b0f198e9093)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:36.113361 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:36.167756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '529209766f4341a4a9012b0f198e9093'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.168418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '529209766f4341a4a9012b0f198e9093'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.176645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '529209766f4341a4a9012b0f198e9093'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.197258 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '529209766f4341a4a9012b0f198e9093'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.204366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1ccc50b-6daf-46ea-acec-d725c8ab10ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '529209766f4341a4a9012b0f198e9093'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.211523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1177/2344] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:20:36 2022] DELETE /identity/v3/users/529209766f4341a4a9012b0f198e9093 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:36.528796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:36.530279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:36.546564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd5deb-8047-4616-ad57-7b992bbd4fc9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:36.604524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:36.618246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:36.619135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:36.619460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:36.619907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] PATH_INFO: `/v3/projects/c175f4c6bba3472a9f4059b8c7bd0d79` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:36.621246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c175f4c6bba3472a9f4059b8c7bd0d79)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:36.623346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:36.677771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c175f4c6bba3472a9f4059b8c7bd0d79', 'request_id': 'req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.684520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c175f4c6bba3472a9f4059b8c7bd0d79', 'request_id': 'req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.690185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c175f4c6bba3472a9f4059b8c7bd0d79', 'request_id': 'req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.703182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] The token cache is being invalidate because project c175f4c6bba3472a9f4059b8c7bd0d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:20:36.703520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.704184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3b2d64a-cd3e-491d-baf3-5b401a51ceff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:20:36.705735 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1168/2345] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:20:36 2022] DELETE /identity/v3/projects/c175f4c6bba3472a9f4059b8c7bd0d79 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:36.711078 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb871bf6-df38-46b9-b8d8-860fc916bdcb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:36.711407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb871bf6-df38-46b9-b8d8-860fc916bdcb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:36.711744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb871bf6-df38-46b9-b8d8-860fc916bdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:36.731325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-eb871bf6-df38-46b9-b8d8-860fc916bdcb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:36.732566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eb871bf6-df38-46b9-b8d8-860fc916bdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:36.806230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1178/2346] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:36.814478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7342b-28a0-4bd0-85d3-f9eba7c4622e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:36.814772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7342b-28a0-4bd0-85d3-f9eba7c4622e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:36.815082 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7342b-28a0-4bd0-85d3-f9eba7c4622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:36.834492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-69a7342b-28a0-4bd0-85d3-f9eba7c4622e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:36.835778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-69a7342b-28a0-4bd0-85d3-f9eba7c4622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:36.843966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1169/2347] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:36.849129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb871bf6-df38-46b9-b8d8-860fc916bdcb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:36.859213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb871bf6-df38-46b9-b8d8-860fc916bdcb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:36.864959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a01c44-7fc1-46d5-8191-dc8c53886d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:36.865638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a01c44-7fc1-46d5-8191-dc8c53886d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:36.865925 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a01c44-7fc1-46d5-8191-dc8c53886d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:36.866203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a01c44-7fc1-46d5-8191-dc8c53886d38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:36.866939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a01c44-7fc1-46d5-8191-dc8c53886d38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:20:36.867246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a01c44-7fc1-46d5-8191-dc8c53886d38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:36.868897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a01c44-7fc1-46d5-8191-dc8c53886d38 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:36.882267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1179/2348] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:20:36.886932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7342b-28a0-4bd0-85d3-f9eba7c4622e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:36.897120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7342b-28a0-4bd0-85d3-f9eba7c4622e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:36.901961 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c51f6-da3a-4b57-b7cf-f769af311372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:36.902637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c51f6-da3a-4b57-b7cf-f769af311372 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:36.902929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c51f6-da3a-4b57-b7cf-f769af311372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:36.903215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c51f6-da3a-4b57-b7cf-f769af311372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:36.903964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c51f6-da3a-4b57-b7cf-f769af311372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:36.905924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c51f6-da3a-4b57-b7cf-f769af311372 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:36.934239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1170/2349] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:20:36 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:20:36.940345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a01c44-7fc1-46d5-8191-dc8c53886d38 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:36.950638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a01c44-7fc1-46d5-8191-dc8c53886d38 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:37.015103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae64a5-bf6e-4e27-a7de-8264b18a8969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:37.015835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae64a5-bf6e-4e27-a7de-8264b18a8969 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:37.016230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae64a5-bf6e-4e27-a7de-8264b18a8969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:37.016462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae64a5-bf6e-4e27-a7de-8264b18a8969 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:37.017271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae64a5-bf6e-4e27-a7de-8264b18a8969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:37.020423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae64a5-bf6e-4e27-a7de-8264b18a8969 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:37.027670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:37.051951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1180/2350] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:20:36 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:20:37.057177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c51f6-da3a-4b57-b7cf-f769af311372 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:37.071772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c51f6-da3a-4b57-b7cf-f769af311372 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:37.076609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08add47c-b61f-46cc-a3a0-e7a4e7b0d922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:37.077345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08add47c-b61f-46cc-a3a0-e7a4e7b0d922 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:37.077636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08add47c-b61f-46cc-a3a0-e7a4e7b0d922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:37.077923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08add47c-b61f-46cc-a3a0-e7a4e7b0d922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:37.078663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08add47c-b61f-46cc-a3a0-e7a4e7b0d922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:37.080385 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08add47c-b61f-46cc-a3a0-e7a4e7b0d922 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:37.099797 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1171/2351] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:37.104064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae64a5-bf6e-4e27-a7de-8264b18a8969 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:37.114645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae64a5-bf6e-4e27-a7de-8264b18a8969 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:37.119908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6b44b-158a-46b3-9707-9398ad4f9574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:37.120590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b44b-158a-46b3-9707-9398ad4f9574 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:37.120879 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b44b-158a-46b3-9707-9398ad4f9574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:37.121159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b44b-158a-46b3-9707-9398ad4f9574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:37.121898 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6b44b-158a-46b3-9707-9398ad4f9574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:37.123541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6b44b-158a-46b3-9707-9398ad4f9574 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:37.134592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1181/2352] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:20:37.139009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08add47c-b61f-46cc-a3a0-e7a4e7b0d922 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:37.150635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08add47c-b61f-46cc-a3a0-e7a4e7b0d922 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:37.156094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2823d1-20dc-4001-b800-127f10faed1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:37.157069 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2823d1-20dc-4001-b800-127f10faed1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:37.157391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2823d1-20dc-4001-b800-127f10faed1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:37.157667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2823d1-20dc-4001-b800-127f10faed1e admin admin] PATH_INFO: `/v3/projects/3306d7e5499140b49aa3a76274d45b98/users/9687c32836f34655bc378743881f0791/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:37.174940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2823d1-20dc-4001-b800-127f10faed1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3306d7e5499140b49aa3a76274d45b98, user_id=9687c32836f34655bc378743881f0791, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:37.177843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2823d1-20dc-4001-b800-127f10faed1e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:37.194045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1172/2353] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:20:37 2022] PUT /identity/v3/projects/3306d7e5499140b49aa3a76274d45b98/users/9687c32836f34655bc378743881f0791/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:20:37.206240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8909d-e861-4626-9609-8ddaed9f442b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:37.207353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8909d-e861-4626-9609-8ddaed9f442b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:37.207353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8909d-e861-4626-9609-8ddaed9f442b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:37.239253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d8e8909d-e861-4626-9609-8ddaed9f442b None None] MFA Rules not processed for user `9687c32836f34655bc378743881f0791`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:20:37.241379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d8e8909d-e861-4626-9609-8ddaed9f442b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:37.309161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1182/2354] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:20:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:20:37.322249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2823d1-20dc-4001-b800-127f10faed1e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:37.324919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6d2823d1-20dc-4001-b800-127f10faed1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:37.391926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2823d1-20dc-4001-b800-127f10faed1e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:37.397673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8acb9c-dd54-4a88-9b97-a4f0f28fb951 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92fc25f9ad7c49c48a909d2d92ea0a66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:37.398438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8acb9c-dd54-4a88-9b97-a4f0f28fb951 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:37.398736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8acb9c-dd54-4a88-9b97-a4f0f28fb951 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:37.399023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8acb9c-dd54-4a88-9b97-a4f0f28fb951 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:37.408599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8acb9c-dd54-4a88-9b97-a4f0f28fb951 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:37.410231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8acb9c-dd54-4a88-9b97-a4f0f28fb951 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:37.420388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1173/2355] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:20:47.305971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:47.416361 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:20:54.183565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8909d-e861-4626-9609-8ddaed9f442b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:20:54.185208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d8e8909d-e861-4626-9609-8ddaed9f442b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:54.273721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8909d-e861-4626-9609-8ddaed9f442b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:20:54.281201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8baab4-b13d-41fa-93a3-7036af345d55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:20:54.282236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8baab4-b13d-41fa-93a3-7036af345d55 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:20:54.282731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8baab4-b13d-41fa-93a3-7036af345d55 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:20:54.283322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8baab4-b13d-41fa-93a3-7036af345d55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:20:54.286952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9d8baab4-b13d-41fa-93a3-7036af345d55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:20:54.367883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8baab4-b13d-41fa-93a3-7036af345d55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:20:54.369442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8baab4-b13d-41fa-93a3-7036af345d55 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:20:54.382495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1183/2356] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:20:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:21:04.374829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=56 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:04.529866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8acb9c-dd54-4a88-9b97-a4f0f28fb951 service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:04.592279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8acb9c-dd54-4a88-9b97-a4f0f28fb951 service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:04.597581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:04.598453 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:04.598734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:04.599006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] PATH_INFO: `/v3/users/9687c32836f34655bc378743881f0791` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:04.600223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9687c32836f34655bc378743881f0791)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:04.602228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:04.648503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9687c32836f34655bc378743881f0791'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:04.655659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9687c32836f34655bc378743881f0791'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:04.662954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9687c32836f34655bc378743881f0791'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:04.679574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9687c32836f34655bc378743881f0791'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:04.680312 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9687c32836f34655bc378743881f0791'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:04.686521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1174/2357] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:04 2022] DELETE /identity/v3/users/9687c32836f34655bc378743881f0791 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:04.691092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64141f93-4cb4-4a65-8d5d-a20df76cf7ad None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:04.691415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64141f93-4cb4-4a65-8d5d-a20df76cf7ad None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:04.691696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64141f93-4cb4-4a65-8d5d-a20df76cf7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:04.710613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-64141f93-4cb4-4a65-8d5d-a20df76cf7ad None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:04.711914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-64141f93-4cb4-4a65-8d5d-a20df76cf7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:04.780327 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1184/2358] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:04.790831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:04.792318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:04.849965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe582-b43b-4055-8661-016e032d987a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:04.853643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2b59f-e2a2-4383-83df-4a65bb15f406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:04.854295 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2b59f-e2a2-4383-83df-4a65bb15f406 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:04.855167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2b59f-e2a2-4383-83df-4a65bb15f406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:04.855167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2b59f-e2a2-4383-83df-4a65bb15f406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:04.865263 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2b59f-e2a2-4383-83df-4a65bb15f406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:04.867425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2b59f-e2a2-4383-83df-4a65bb15f406 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:04.878139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1175/2359] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:05.322574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64141f93-4cb4-4a65-8d5d-a20df76cf7ad None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:05.324049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-64141f93-4cb4-4a65-8d5d-a20df76cf7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:05.339630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64141f93-4cb4-4a65-8d5d-a20df76cf7ad None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:05.346204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:05.347161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:05.347854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:05.348146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] PATH_INFO: `/v3/projects/3306d7e5499140b49aa3a76274d45b98` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:05.349402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3306d7e5499140b49aa3a76274d45b98)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:05.351408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:05.419437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3306d7e5499140b49aa3a76274d45b98', 'request_id': 'req-84a4df57-374e-4df1-9ae5-e98fbacf2db0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:05.427164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3306d7e5499140b49aa3a76274d45b98', 'request_id': 'req-84a4df57-374e-4df1-9ae5-e98fbacf2db0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:05.439386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3306d7e5499140b49aa3a76274d45b98', 'request_id': 'req-84a4df57-374e-4df1-9ae5-e98fbacf2db0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:05.453956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] The token cache is being invalidate because project 3306d7e5499140b49aa3a76274d45b98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:05.454253 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:05.455316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84a4df57-374e-4df1-9ae5-e98fbacf2db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:05.457501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1185/2360] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:05 2022] DELETE /identity/v3/projects/3306d7e5499140b49aa3a76274d45b98 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:05.464788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca83e5-bbe6-49b6-a3e3-786a86361f5b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:05.465509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca83e5-bbe6-49b6-a3e3-786a86361f5b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:05.465870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca83e5-bbe6-49b6-a3e3-786a86361f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:05.487891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eaca83e5-bbe6-49b6-a3e3-786a86361f5b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:05.491898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eaca83e5-bbe6-49b6-a3e3-786a86361f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:05.518093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:05.579836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1176/2361] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:05.588356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18060140-3bd0-4a55-9813-3db1f7191541 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:05.588917 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18060140-3bd0-4a55-9813-3db1f7191541 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:05.589380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18060140-3bd0-4a55-9813-3db1f7191541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:05.644125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-18060140-3bd0-4a55-9813-3db1f7191541 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:05.644125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-18060140-3bd0-4a55-9813-3db1f7191541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:05.659775 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1186/2362] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:05.662762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca83e5-bbe6-49b6-a3e3-786a86361f5b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:05.676271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca83e5-bbe6-49b6-a3e3-786a86361f5b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:05.681565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368a593-3c83-429a-ab25-b3aaabe6c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:05.682171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368a593-3c83-429a-ab25-b3aaabe6c525 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:05.682456 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368a593-3c83-429a-ab25-b3aaabe6c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:05.682735 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368a593-3c83-429a-ab25-b3aaabe6c525 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:05.683506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368a593-3c83-429a-ab25-b3aaabe6c525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:21:05.683865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368a593-3c83-429a-ab25-b3aaabe6c525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:05.685531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368a593-3c83-429a-ab25-b3aaabe6c525 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:05.709041 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1177/2363] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:21:05.714107 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18060140-3bd0-4a55-9813-3db1f7191541 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:05.728837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18060140-3bd0-4a55-9813-3db1f7191541 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:05.735376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7644b1-5c91-4a17-af7e-c41f8fa5919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:05.736712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7644b1-5c91-4a17-af7e-c41f8fa5919c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:05.737516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7644b1-5c91-4a17-af7e-c41f8fa5919c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:05.737810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7644b1-5c91-4a17-af7e-c41f8fa5919c admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:05.738832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7644b1-5c91-4a17-af7e-c41f8fa5919c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:05.741515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7644b1-5c91-4a17-af7e-c41f8fa5919c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:05.774727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1187/2364] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:05 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:05.779813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368a593-3c83-429a-ab25-b3aaabe6c525 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:05.794544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368a593-3c83-429a-ab25-b3aaabe6c525 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:05.867836 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20188e1e-ac66-4dcc-ab37-de7083313182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:05.868445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20188e1e-ac66-4dcc-ab37-de7083313182 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:05.868737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20188e1e-ac66-4dcc-ab37-de7083313182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:05.869024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20188e1e-ac66-4dcc-ab37-de7083313182 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:05.869807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20188e1e-ac66-4dcc-ab37-de7083313182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:05.872126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20188e1e-ac66-4dcc-ab37-de7083313182 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:05.899675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1178/2365] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:05 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:05.900956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7644b1-5c91-4a17-af7e-c41f8fa5919c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:05.914435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7644b1-5c91-4a17-af7e-c41f8fa5919c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:05.919623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2c9ca-fed7-4646-a29b-d805f144c5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:05.920335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2c9ca-fed7-4646-a29b-d805f144c5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:05.920626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2c9ca-fed7-4646-a29b-d805f144c5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:05.920908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2c9ca-fed7-4646-a29b-d805f144c5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:05.921650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2c9ca-fed7-4646-a29b-d805f144c5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:05.923473 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2c9ca-fed7-4646-a29b-d805f144c5d8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:05.936152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1188/2366] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:05.939973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20188e1e-ac66-4dcc-ab37-de7083313182 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:05.953503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20188e1e-ac66-4dcc-ab37-de7083313182 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:05.963579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86c935-3c2d-4038-8125-28637dbe7975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:05.964521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86c935-3c2d-4038-8125-28637dbe7975 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:05.964958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86c935-3c2d-4038-8125-28637dbe7975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:05.965377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86c935-3c2d-4038-8125-28637dbe7975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:05.966136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86c935-3c2d-4038-8125-28637dbe7975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:05.967919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86c935-3c2d-4038-8125-28637dbe7975 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:05.984426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1179/2367] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:05.989838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2c9ca-fed7-4646-a29b-d805f144c5d8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:06.001304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2c9ca-fed7-4646-a29b-d805f144c5d8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:06.006497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f039372-33f7-4ac3-b120-40f1789009c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:06.007287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f039372-33f7-4ac3-b120-40f1789009c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:06.007595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f039372-33f7-4ac3-b120-40f1789009c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:06.007881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f039372-33f7-4ac3-b120-40f1789009c4 admin admin] PATH_INFO: `/v3/projects/04d58bceb8414b02a1bbaec0db07d85e/users/1a299663b56b411b9fc910ddf47103e5/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:06.030035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f039372-33f7-4ac3-b120-40f1789009c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d58bceb8414b02a1bbaec0db07d85e, user_id=1a299663b56b411b9fc910ddf47103e5, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:06.032749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f039372-33f7-4ac3-b120-40f1789009c4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:06.048653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1189/2368] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:05 2022] PUT /identity/v3/projects/04d58bceb8414b02a1bbaec0db07d85e/users/1a299663b56b411b9fc910ddf47103e5/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:06.054304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a1ee4-f258-4655-ae83-e5b477e8fb17 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:06.054609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a1ee4-f258-4655-ae83-e5b477e8fb17 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:06.054882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a1ee4-f258-4655-ae83-e5b477e8fb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:06.072812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ca5a1ee4-f258-4655-ae83-e5b477e8fb17 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:06.074023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca5a1ee4-f258-4655-ae83-e5b477e8fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:06.172574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1180/2369] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:06.184512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f039372-33f7-4ac3-b120-40f1789009c4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:06.187428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3f039372-33f7-4ac3-b120-40f1789009c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:06.224351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:06.271993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f039372-33f7-4ac3-b120-40f1789009c4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:06.279068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1d157-e73d-4e02-8fcb-b9b1201f2e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:06.279868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1d157-e73d-4e02-8fcb-b9b1201f2e26 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:06.280778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1d157-e73d-4e02-8fcb-b9b1201f2e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:06.281149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1d157-e73d-4e02-8fcb-b9b1201f2e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:06.306303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1d157-e73d-4e02-8fcb-b9b1201f2e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:06.306303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1d157-e73d-4e02-8fcb-b9b1201f2e26 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:06.317732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1190/2370] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:10.717778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d2289-bb2f-4cc8-88fa-568175f03ee6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:10.718359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d2289-bb2f-4cc8-88fa-568175f03ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:10.718359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d2289-bb2f-4cc8-88fa-568175f03ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:10.757953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e84d2289-bb2f-4cc8-88fa-568175f03ee6 None None] MFA Rules not processed for user `1a299663b56b411b9fc910ddf47103e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:10.759364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e84d2289-bb2f-4cc8-88fa-568175f03ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:10.781936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:10.848878 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1181/2371] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:10.854720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1d157-e73d-4e02-8fcb-b9b1201f2e26 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:10.866338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1d157-e73d-4e02-8fcb-b9b1201f2e26 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:10.871464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d41a83-0a4c-4cfd-9649-0dc6dabaf804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:10.872254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d41a83-0a4c-4cfd-9649-0dc6dabaf804 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:10.872542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d41a83-0a4c-4cfd-9649-0dc6dabaf804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:10.872818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d41a83-0a4c-4cfd-9649-0dc6dabaf804 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:10.873519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d41a83-0a4c-4cfd-9649-0dc6dabaf804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:10.875811 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d41a83-0a4c-4cfd-9649-0dc6dabaf804 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:10.907432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1191/2372] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:10 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:10.910020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d2289-bb2f-4cc8-88fa-568175f03ee6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:10.922833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d2289-bb2f-4cc8-88fa-568175f03ee6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:10.996600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0fa2f-1006-4bc7-98c4-f8b335597633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:10.997395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0fa2f-1006-4bc7-98c4-f8b335597633 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:10.997395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0fa2f-1006-4bc7-98c4-f8b335597633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:10.997646 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0fa2f-1006-4bc7-98c4-f8b335597633 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:10.998273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0fa2f-1006-4bc7-98c4-f8b335597633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:11.000831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0fa2f-1006-4bc7-98c4-f8b335597633 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:11.030530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1182/2373] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:10 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:21:11.032013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d41a83-0a4c-4cfd-9649-0dc6dabaf804 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:11.044204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d41a83-0a4c-4cfd-9649-0dc6dabaf804 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:11.052227 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c738e7-827c-4aaf-8e40-a0ba611cb8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:11.054092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c738e7-827c-4aaf-8e40-a0ba611cb8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:11.054347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c738e7-827c-4aaf-8e40-a0ba611cb8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:11.054632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c738e7-827c-4aaf-8e40-a0ba611cb8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:11.055418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c738e7-827c-4aaf-8e40-a0ba611cb8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:11.057202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c738e7-827c-4aaf-8e40-a0ba611cb8d7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:11.076583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1192/2374] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:11.080693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0fa2f-1006-4bc7-98c4-f8b335597633 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:11.095501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0fa2f-1006-4bc7-98c4-f8b335597633 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:11.100612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a7057-1d17-4fd4-841c-aa4c56f6fb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:11.101782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a7057-1d17-4fd4-841c-aa4c56f6fb35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:11.101926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a7057-1d17-4fd4-841c-aa4c56f6fb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:11.102166 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a7057-1d17-4fd4-841c-aa4c56f6fb35 admin admin] PATH_INFO: `/v3/projects/4ed3871a9ef44759abf5a216c756e1df/users/e253928c11e64bb5b187f4f3dd8cf95f/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:11.126985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a7057-1d17-4fd4-841c-aa4c56f6fb35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed3871a9ef44759abf5a216c756e1df, user_id=e253928c11e64bb5b187f4f3dd8cf95f, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:11.130370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a7057-1d17-4fd4-841c-aa4c56f6fb35 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:11.151627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1183/2375] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:11 2022] PUT /identity/v3/projects/4ed3871a9ef44759abf5a216c756e1df/users/e253928c11e64bb5b187f4f3dd8cf95f/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:21:11.159912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c738e7-827c-4aaf-8e40-a0ba611cb8d7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:11.183551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c738e7-827c-4aaf-8e40-a0ba611cb8d7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:11.262117 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c826f-95b0-4455-8204-fb72cfb78bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:11.263200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c826f-95b0-4455-8204-fb72cfb78bee admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:11.263754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c826f-95b0-4455-8204-fb72cfb78bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:11.264279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c826f-95b0-4455-8204-fb72cfb78bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:11.265779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c826f-95b0-4455-8204-fb72cfb78bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:11.269267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c826f-95b0-4455-8204-fb72cfb78bee admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:11.290383 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1193/2376] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:11.295794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a7057-1d17-4fd4-841c-aa4c56f6fb35 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:11.312871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a7057-1d17-4fd4-841c-aa4c56f6fb35 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:11.317939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92477a33-b607-4d82-8134-daa12dc3e1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:11.318944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92477a33-b607-4d82-8134-daa12dc3e1b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:11.319308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92477a33-b607-4d82-8134-daa12dc3e1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:11.319553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92477a33-b607-4d82-8134-daa12dc3e1b8 admin admin] PATH_INFO: `/v3/projects/4ed3871a9ef44759abf5a216c756e1df/users/e253928c11e64bb5b187f4f3dd8cf95f/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:11.321898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92477a33-b607-4d82-8134-daa12dc3e1b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed3871a9ef44759abf5a216c756e1df, user_id=e253928c11e64bb5b187f4f3dd8cf95f, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:11.324710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92477a33-b607-4d82-8134-daa12dc3e1b8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:11.345112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1184/2377] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:11 2022] PUT /identity/v3/projects/4ed3871a9ef44759abf5a216c756e1df/users/e253928c11e64bb5b187f4f3dd8cf95f/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:16.132164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c10f0e-a1be-4185-a5c0-226ee08d64d9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:16.132693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c10f0e-a1be-4185-a5c0-226ee08d64d9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:16.132779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c10f0e-a1be-4185-a5c0-226ee08d64d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:16.163867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-72c10f0e-a1be-4185-a5c0-226ee08d64d9 None None] MFA Rules not processed for user `e253928c11e64bb5b187f4f3dd8cf95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:16.165115 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-72c10f0e-a1be-4185-a5c0-226ee08d64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:16.246629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1194/2378] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:16.260785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92477a33-b607-4d82-8134-daa12dc3e1b8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:16.268188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92477a33-b607-4d82-8134-daa12dc3e1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:16.354611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92477a33-b607-4d82-8134-daa12dc3e1b8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:16.360111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcbfbb-53c7-4907-b155-3b79c3b41be1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:16.360765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcbfbb-53c7-4907-b155-3b79c3b41be1 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:16.361055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcbfbb-53c7-4907-b155-3b79c3b41be1 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:16.361336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcbfbb-53c7-4907-b155-3b79c3b41be1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:16.367369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:16.422395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcbfbb-53c7-4907-b155-3b79c3b41be1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:16.423893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcbfbb-53c7-4907-b155-3b79c3b41be1 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:16.433848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1185/2379] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:22.714762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3699e-581f-4f44-a23f-9cd2cde4cbd9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:22.715576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3699e-581f-4f44-a23f-9cd2cde4cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:22.715576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3699e-581f-4f44-a23f-9cd2cde4cbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:22.737215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-21e3699e-581f-4f44-a23f-9cd2cde4cbd9 None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:22.738601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-21e3699e-581f-4f44-a23f-9cd2cde4cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:22.745892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:22.831388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1195/2380] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:21:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:22.882600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcbfbb-53c7-4907-b155-3b79c3b41be1 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:22.884173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6dbcbfbb-53c7-4907-b155-3b79c3b41be1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:22.995592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcbfbb-53c7-4907-b155-3b79c3b41be1 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:23.006484 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba65c3-3feb-4395-89e7-2ec2cf0a16c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:23.007606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba65c3-3feb-4395-89e7-2ec2cf0a16c3 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:23.008055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba65c3-3feb-4395-89e7-2ec2cf0a16c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:23.008504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba65c3-3feb-4395-89e7-2ec2cf0a16c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:23.047667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba65c3-3feb-4395-89e7-2ec2cf0a16c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:23.050466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba65c3-3feb-4395-89e7-2ec2cf0a16c3 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:23.068127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1186/2381] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:21:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:21:30.753102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3699e-581f-4f44-a23f-9cd2cde4cbd9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:30.865189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3699e-581f-4f44-a23f-9cd2cde4cbd9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:30.869471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:30.870715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:30.871165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:30.871612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:30.874785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:31.002953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:31.009222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:31.023109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1196/2382] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:33.058388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=56 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:36.282189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba65c3-3feb-4395-89e7-2ec2cf0a16c3 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:36.285162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bdba65c3-3feb-4395-89e7-2ec2cf0a16c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:36.368892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:36.370557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:36.388243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf9771-0589-407b-b90f-afb4a0efed59 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:36.393820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:36.394738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:36.395028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:36.395346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] PATH_INFO: `/v3/users/691877dce1024aeda61629073d74602b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:36.396718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=691877dce1024aeda61629073d74602b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:36.398846 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:36.410058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba65c3-3feb-4395-89e7-2ec2cf0a16c3 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:36.416685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119be29e-53ef-4b88-9158-66fc74f99120 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:36.417351 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119be29e-53ef-4b88-9158-66fc74f99120 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:36.417638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119be29e-53ef-4b88-9158-66fc74f99120 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:36.417915 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119be29e-53ef-4b88-9158-66fc74f99120 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:36.420586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-119be29e-53ef-4b88-9158-66fc74f99120 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:36.438428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119be29e-53ef-4b88-9158-66fc74f99120 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:36.440365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119be29e-53ef-4b88-9158-66fc74f99120 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:36.453019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1187/2383] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:36.457215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691877dce1024aeda61629073d74602b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.457960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691877dce1024aeda61629073d74602b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.468244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691877dce1024aeda61629073d74602b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.478279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:36.491384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691877dce1024aeda61629073d74602b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.498304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691877dce1024aeda61629073d74602b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.505462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1197/2384] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:36 2022] DELETE /identity/v3/users/691877dce1024aeda61629073d74602b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:36.732670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119be29e-53ef-4b88-9158-66fc74f99120 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:36.774061 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119be29e-53ef-4b88-9158-66fc74f99120 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:36.828507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:36.831465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:36.862977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:36.864098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:36.864817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:36.865258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] PATH_INFO: `/v3/projects/28836b7cb9c646aaaa1bb1c1fae68128` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:36.866863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28836b7cb9c646aaaa1bb1c1fae68128)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:36.869101 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:36.895627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e232c-9722-4824-8d79-99237482d446 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:36.899520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38347e1-932e-435a-a248-725dc3563793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:36.900273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38347e1-932e-435a-a248-725dc3563793 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:36.900637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38347e1-932e-435a-a248-725dc3563793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:36.900952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38347e1-932e-435a-a248-725dc3563793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:36.903109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f38347e1-932e-435a-a248-725dc3563793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:36.916347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38347e1-932e-435a-a248-725dc3563793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:36.917884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38347e1-932e-435a-a248-725dc3563793 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:36.924273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28836b7cb9c646aaaa1bb1c1fae68128', 'request_id': 'req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.932303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1198/2385] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:21:36.934216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28836b7cb9c646aaaa1bb1c1fae68128', 'request_id': 'req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.942966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28836b7cb9c646aaaa1bb1c1fae68128', 'request_id': 'req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.958910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] The token cache is being invalidate because project 28836b7cb9c646aaaa1bb1c1fae68128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:36.959232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.960014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d7cc0ad7-4aa3-4a0a-8eca-f61976b1e240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:36.965155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1188/2386] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:36 2022] DELETE /identity/v3/projects/28836b7cb9c646aaaa1bb1c1fae68128 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:36.966681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4fe0f-3ca8-4c34-b469-b9d67610e707 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:36.966979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4fe0f-3ca8-4c34-b469-b9d67610e707 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:36.967365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4fe0f-3ca8-4c34-b469-b9d67610e707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:36.989783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cbb4fe0f-3ca8-4c34-b469-b9d67610e707 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:36.991065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cbb4fe0f-3ca8-4c34-b469-b9d67610e707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:37.083320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1199/2387] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:37.091630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da591ed-79c4-48b2-8f0e-df2390a3b74e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.091920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da591ed-79c4-48b2-8f0e-df2390a3b74e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.092138 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da591ed-79c4-48b2-8f0e-df2390a3b74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.112601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7da591ed-79c4-48b2-8f0e-df2390a3b74e None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:37.113884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7da591ed-79c4-48b2-8f0e-df2390a3b74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:37.129420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1189/2388] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:37.135434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4fe0f-3ca8-4c34-b469-b9d67610e707 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:37.147516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4fe0f-3ca8-4c34-b469-b9d67610e707 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:37.155304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e5056-9178-4247-abef-aa7ecc634a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:37.155304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e5056-9178-4247-abef-aa7ecc634a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.155304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e5056-9178-4247-abef-aa7ecc634a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.155960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e5056-9178-4247-abef-aa7ecc634a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.156396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e5056-9178-4247-abef-aa7ecc634a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:21:37.156603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e5056-9178-4247-abef-aa7ecc634a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:37.158225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e5056-9178-4247-abef-aa7ecc634a7d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:37.178797 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1200/2389] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:21:37.184058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da591ed-79c4-48b2-8f0e-df2390a3b74e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:37.196975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da591ed-79c4-48b2-8f0e-df2390a3b74e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:37.204663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20337a-9fd3-4380-a03b-9e029c108361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:37.205346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20337a-9fd3-4380-a03b-9e029c108361 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.205630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20337a-9fd3-4380-a03b-9e029c108361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.205907 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20337a-9fd3-4380-a03b-9e029c108361 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.206607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20337a-9fd3-4380-a03b-9e029c108361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:37.208565 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20337a-9fd3-4380-a03b-9e029c108361 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:37.221186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:37.241550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1190/2390] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:37 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:37.246432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e5056-9178-4247-abef-aa7ecc634a7d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:37.257648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e5056-9178-4247-abef-aa7ecc634a7d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:37.334003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c784e38-2568-435d-8673-f9507991d3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:37.334692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c784e38-2568-435d-8673-f9507991d3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.335410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c784e38-2568-435d-8673-f9507991d3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.335410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c784e38-2568-435d-8673-f9507991d3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.335967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c784e38-2568-435d-8673-f9507991d3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:37.337988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c784e38-2568-435d-8673-f9507991d3b9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:37.355798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:37.365209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1201/2391] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:37 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:37.367593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20337a-9fd3-4380-a03b-9e029c108361 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:37.379081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20337a-9fd3-4380-a03b-9e029c108361 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:37.386462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb3269-fd25-4c29-8077-94ad02d7bc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:37.387732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb3269-fd25-4c29-8077-94ad02d7bc24 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.388198 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb3269-fd25-4c29-8077-94ad02d7bc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.388629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb3269-fd25-4c29-8077-94ad02d7bc24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.389538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb3269-fd25-4c29-8077-94ad02d7bc24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:37.391702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb3269-fd25-4c29-8077-94ad02d7bc24 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:37.407287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1191/2392] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:37.415020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c784e38-2568-435d-8673-f9507991d3b9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:37.434135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c784e38-2568-435d-8673-f9507991d3b9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:37.443677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488ed11-c9f4-4d8d-a579-acf5feea61fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:37.444928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488ed11-c9f4-4d8d-a579-acf5feea61fa admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.445520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488ed11-c9f4-4d8d-a579-acf5feea61fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.446417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488ed11-c9f4-4d8d-a579-acf5feea61fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.448271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488ed11-c9f4-4d8d-a579-acf5feea61fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:37.451459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488ed11-c9f4-4d8d-a579-acf5feea61fa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:37.465721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1202/2393] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:37.466972 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb3269-fd25-4c29-8077-94ad02d7bc24 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:37.480139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb3269-fd25-4c29-8077-94ad02d7bc24 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:37.487503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7563292-ae87-4cb4-bda0-43b9d12a57ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:37.489092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7563292-ae87-4cb4-bda0-43b9d12a57ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.489713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7563292-ae87-4cb4-bda0-43b9d12a57ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.490067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7563292-ae87-4cb4-bda0-43b9d12a57ef admin admin] PATH_INFO: `/v3/projects/07a139ca32b644f9af419289061346dc/users/be8df4382111426980dcdf8239e49ecc/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.500626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e099f6a-dbb8-4d77-b83c-76e3c1be7cd6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.501124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e099f6a-dbb8-4d77-b83c-76e3c1be7cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.502671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e099f6a-dbb8-4d77-b83c-76e3c1be7cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.507609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7563292-ae87-4cb4-bda0-43b9d12a57ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a139ca32b644f9af419289061346dc, user_id=be8df4382111426980dcdf8239e49ecc, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:37.510487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7563292-ae87-4cb4-bda0-43b9d12a57ef admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:37.525197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0e099f6a-dbb8-4d77-b83c-76e3c1be7cd6 None None] MFA Rules not processed for user `1a299663b56b411b9fc910ddf47103e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:37.526068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1192/2394] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:37 2022] PUT /identity/v3/projects/07a139ca32b644f9af419289061346dc/users/be8df4382111426980dcdf8239e49ecc/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:37.527096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0e099f6a-dbb8-4d77-b83c-76e3c1be7cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:37.537010 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee1b21-ca51-46e8-857b-5cfce9b8fcb6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.537302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee1b21-ca51-46e8-857b-5cfce9b8fcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.537585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee1b21-ca51-46e8-857b-5cfce9b8fcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.569852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5cee1b21-ca51-46e8-857b-5cfce9b8fcb6 None None] MFA Rules not processed for user `be8df4382111426980dcdf8239e49ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:37.571756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5cee1b21-ca51-46e8-857b-5cfce9b8fcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:37.612484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1203/2395] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:37.659717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1193/2396] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:37.670430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e099f6a-dbb8-4d77-b83c-76e3c1be7cd6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:37.677342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0e099f6a-dbb8-4d77-b83c-76e3c1be7cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:37.772918 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e099f6a-dbb8-4d77-b83c-76e3c1be7cd6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:37.780958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6d3fd-1a6c-4d35-aaeb-fd6a00defecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:37.781645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6d3fd-1a6c-4d35-aaeb-fd6a00defecf service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:37.782058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6d3fd-1a6c-4d35-aaeb-fd6a00defecf service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:37.782344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6d3fd-1a6c-4d35-aaeb-fd6a00defecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:37.791701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6d3fd-1a6c-4d35-aaeb-fd6a00defecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:37.793421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6d3fd-1a6c-4d35-aaeb-fd6a00defecf service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:37.809296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1204/2397] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:40.856190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee1b21-ca51-46e8-857b-5cfce9b8fcb6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:40.972599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee1b21-ca51-46e8-857b-5cfce9b8fcb6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:40.983017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:40.984399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:40.984990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:40.985430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] PATH_INFO: `/v3/users/be8df4382111426980dcdf8239e49ecc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:40.987094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be8df4382111426980dcdf8239e49ecc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:40.990754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:41.127187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be8df4382111426980dcdf8239e49ecc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:41.130347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:41.144573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be8df4382111426980dcdf8239e49ecc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:41.155712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be8df4382111426980dcdf8239e49ecc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:41.178814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be8df4382111426980dcdf8239e49ecc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:41.179631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be8df4382111426980dcdf8239e49ecc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:41.206236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1194/2398] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:40 2022] DELETE /identity/v3/users/be8df4382111426980dcdf8239e49ecc => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:41.211096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13885fa-bffb-4656-a73b-b57cc8217286 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:41.211638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13885fa-bffb-4656-a73b-b57cc8217286 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:41.211874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13885fa-bffb-4656-a73b-b57cc8217286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:41.234965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a13885fa-bffb-4656-a73b-b57cc8217286 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:41.236716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a13885fa-bffb-4656-a73b-b57cc8217286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:41.390285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1205/2399] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:41.418770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:41.420535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:41.446016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24131b-d7d5-48cf-a740-8b7e30fb0917 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:41.446313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24131b-d7d5-48cf-a740-8b7e30fb0917 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:41.446603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24131b-d7d5-48cf-a740-8b7e30fb0917 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:41.448229 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1206/2400] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 22 11:21:41.456050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0047c-c480-424a-a24a-836d9ad75069 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:41.456608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0047c-c480-424a-a24a-836d9ad75069 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:41.457052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0047c-c480-424a-a24a-836d9ad75069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:41.539384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df29bf2-fc52-41cd-875b-644ad9a66b50 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:41.543883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bccc2d-38c9-44fd-bbc5-c4a3db7f45f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:41.544574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bccc2d-38c9-44fd-bbc5-c4a3db7f45f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:41.545035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bccc2d-38c9-44fd-bbc5-c4a3db7f45f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:41.545338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bccc2d-38c9-44fd-bbc5-c4a3db7f45f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:41.558391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bccc2d-38c9-44fd-bbc5-c4a3db7f45f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:41.560248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bccc2d-38c9-44fd-bbc5-c4a3db7f45f4 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:41.574530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1195/2401] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:41.616636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b3d0047c-c480-424a-a24a-836d9ad75069 None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:41.617883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b3d0047c-c480-424a-a24a-836d9ad75069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:41.633346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:41.702913 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1207/2402] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:21:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:41.722663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bccc2d-38c9-44fd-bbc5-c4a3db7f45f4 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:41.725046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a8bccc2d-38c9-44fd-bbc5-c4a3db7f45f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:41.807675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bccc2d-38c9-44fd-bbc5-c4a3db7f45f4 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:41.811958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e2ef9-324d-44ee-9b6b-a1fd4eb5facd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92fc25f9ad7c49c48a909d2d92ea0a66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:41.812657 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e2ef9-324d-44ee-9b6b-a1fd4eb5facd service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:41.812979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e2ef9-324d-44ee-9b6b-a1fd4eb5facd service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:41.813223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e2ef9-324d-44ee-9b6b-a1fd4eb5facd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:41.828289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e2ef9-324d-44ee-9b6b-a1fd4eb5facd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:41.830363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e2ef9-324d-44ee-9b6b-a1fd4eb5facd service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:41.846343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1196/2403] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:41.917218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0047c-c480-424a-a24a-836d9ad75069 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:41.918695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b3d0047c-c480-424a-a24a-836d9ad75069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:41.944379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0047c-c480-424a-a24a-836d9ad75069 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:41.954660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:41.956359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1f5b-652b-4b55-91f8-563a5aecd886 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:41.956861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1f5b-652b-4b55-91f8-563a5aecd886 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:41.956861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1f5b-652b-4b55-91f8-563a5aecd886 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:41.958097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1197/2404] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:21:41.960362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:41.960900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:41.961404 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] PATH_INFO: `/v3/users/da75b78b267f47e8aaaafb41e3acb61a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:41.963550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=da75b78b267f47e8aaaafb41e3acb61a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:41.963704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264feed1-c530-4c82-b540-e3fc1656c734 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:41.963871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264feed1-c530-4c82-b540-e3fc1656c734 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:41.964346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264feed1-c530-4c82-b540-e3fc1656c734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:41.967553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:42.013865 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-264feed1-c530-4c82-b540-e3fc1656c734 None None] MFA Rules not processed for user `0a0ce1804b514b42845ab33c7bfee309`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:42.015443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-264feed1-c530-4c82-b540-e3fc1656c734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.026672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1198/2405] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:21:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:42.046701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264feed1-c530-4c82-b540-e3fc1656c734 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:42.053612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da75b78b267f47e8aaaafb41e3acb61a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.054512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da75b78b267f47e8aaaafb41e3acb61a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.057821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264feed1-c530-4c82-b540-e3fc1656c734 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:42.062306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:42.063235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.063566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.063926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.066022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.066414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da75b78b267f47e8aaaafb41e3acb61a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.080965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da75b78b267f47e8aaaafb41e3acb61a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.082125 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:42.083733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:42.089209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8a31c370-caf2-4cbe-871a-b1f82533a7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da75b78b267f47e8aaaafb41e3acb61a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.094044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1199/2406] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:21:42.103200 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1208/2407] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:41 2022] DELETE /identity/v3/users/da75b78b267f47e8aaaafb41e3acb61a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:42.200329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:42.204759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.231117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78867ff5-4a4b-4843-855a-fa971e636f4c service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:42.254228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61532d33-e574-4b5e-96c0-8ed13ab47672 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.254815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61532d33-e574-4b5e-96c0-8ed13ab47672 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.255027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61532d33-e574-4b5e-96c0-8ed13ab47672 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.256505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1209/2408] 10.222.0.20 () {54 vars in 921 bytes} [Fri Jul 22 11:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 22 11:21:42.263805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fa953-22f4-4a2e-a007-e40191f861db None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.264165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fa953-22f4-4a2e-a007-e40191f861db None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.264746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fa953-22f4-4a2e-a007-e40191f861db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.285079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d16fa953-22f4-4a2e-a007-e40191f861db None None] MFA Rules not processed for user `c51929d0ecb047b4bdc405dc86fe7eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:42.286309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d16fa953-22f4-4a2e-a007-e40191f861db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.309939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:42.335606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:42.336850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.337783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.338029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] PATH_INFO: `/v3/projects/07a139ca32b644f9af419289061346dc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.346139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07a139ca32b644f9af419289061346dc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:42.348402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:42.386147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1210/2409] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Jul 22 11:21:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:42.403660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fa953-22f4-4a2e-a007-e40191f861db None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:42.405183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d16fa953-22f4-4a2e-a007-e40191f861db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.414319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07a139ca32b644f9af419289061346dc', 'request_id': 'req-a9758b74-381f-41d8-977a-841f8400f337'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.424973 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07a139ca32b644f9af419289061346dc', 'request_id': 'req-a9758b74-381f-41d8-977a-841f8400f337'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.425213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:42.432211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07a139ca32b644f9af419289061346dc', 'request_id': 'req-a9758b74-381f-41d8-977a-841f8400f337'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.448807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] The token cache is being invalidate because project 07a139ca32b644f9af419289061346dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:42.449144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.449981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9758b74-381f-41d8-977a-841f8400f337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.455379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1200/2410] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:42 2022] DELETE /identity/v3/projects/07a139ca32b644f9af419289061346dc => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:42.457334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbee75-6e3b-4724-a6ae-2077a77b6b89 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.457658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbee75-6e3b-4724-a6ae-2077a77b6b89 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.457927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbee75-6e3b-4724-a6ae-2077a77b6b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.488153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-00cbee75-6e3b-4724-a6ae-2077a77b6b89 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:42.489816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-00cbee75-6e3b-4724-a6ae-2077a77b6b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.490874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fa953-22f4-4a2e-a007-e40191f861db None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:42.566359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be921c6d-9673-4c22-9c5c-2a451c492171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:42.567033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be921c6d-9673-4c22-9c5c-2a451c492171 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.567334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be921c6d-9673-4c22-9c5c-2a451c492171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.567620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be921c6d-9673-4c22-9c5c-2a451c492171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.569531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-be921c6d-9673-4c22-9c5c-2a451c492171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.628988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1201/2411] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:42.630015 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbee75-6e3b-4724-a6ae-2077a77b6b89 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:42.631693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-00cbee75-6e3b-4724-a6ae-2077a77b6b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.652207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbee75-6e3b-4724-a6ae-2077a77b6b89 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:42.657414 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:42.658870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.659367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.659670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] PATH_INFO: `/v3/projects/f50a15db11214fb5a637fb400dcd928f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.660892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50a15db11214fb5a637fb400dcd928f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:42.662831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:42.672483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be921c6d-9673-4c22-9c5c-2a451c492171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:42.674160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be921c6d-9673-4c22-9c5c-2a451c492171 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:42.700875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1211/2412] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:21:42.703295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdbe18-3076-4147-8031-9db94f24ce73 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.705743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdbe18-3076-4147-8031-9db94f24ce73 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.706348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdbe18-3076-4147-8031-9db94f24ce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.748661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5bbdbe18-3076-4147-8031-9db94f24ce73 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:42.751781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5bbdbe18-3076-4147-8031-9db94f24ce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.767943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50a15db11214fb5a637fb400dcd928f', 'request_id': 'req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.775250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50a15db11214fb5a637fb400dcd928f', 'request_id': 'req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.784608 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50a15db11214fb5a637fb400dcd928f', 'request_id': 'req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.824032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] The token cache is being invalidate because project f50a15db11214fb5a637fb400dcd928f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:42.824412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.826292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6313e551-b4c2-4a06-a8d8-2d2e56e35d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:42.836477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1202/2413] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:42 2022] DELETE /identity/v3/projects/f50a15db11214fb5a637fb400dcd928f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:42.839611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e19306-e229-4c06-97fb-a180e25136f6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.840030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e19306-e229-4c06-97fb-a180e25136f6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.840439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e19306-e229-4c06-97fb-a180e25136f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.859241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:42.894029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1212/2414] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:42.895003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdbe18-3076-4147-8031-9db94f24ce73 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:42.895988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a9e19306-e229-4c06-97fb-a180e25136f6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:42.898609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9e19306-e229-4c06-97fb-a180e25136f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:42.909650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdbe18-3076-4147-8031-9db94f24ce73 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:42.922956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1203/2415] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:42.929437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd633db-988a-4cc8-816b-045e9729b529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:42.929437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd633db-988a-4cc8-816b-045e9729b529 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.929437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd633db-988a-4cc8-816b-045e9729b529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.929847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd633db-988a-4cc8-816b-045e9729b529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.930234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd633db-988a-4cc8-816b-045e9729b529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:21:42.930526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd633db-988a-4cc8-816b-045e9729b529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:42.931905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b668f-a25b-46ad-8929-c959ac8723ff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:42.932181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd633db-988a-4cc8-816b-045e9729b529 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:42.932388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b668f-a25b-46ad-8929-c959ac8723ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:42.932774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b668f-a25b-46ad-8929-c959ac8723ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:42.943169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:42.964062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1213/2416] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:21:42.969602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd633db-988a-4cc8-816b-045e9729b529 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:42.987518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd633db-988a-4cc8-816b-045e9729b529 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.003449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefabb52-2ca4-440a-bdc8-c172372ce2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.004008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefabb52-2ca4-440a-bdc8-c172372ce2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.004370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefabb52-2ca4-440a-bdc8-c172372ce2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.004563 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefabb52-2ca4-440a-bdc8-c172372ce2c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.005270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefabb52-2ca4-440a-bdc8-c172372ce2c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.007210 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefabb52-2ca4-440a-bdc8-c172372ce2c8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.010829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-df4b668f-a25b-46ad-8929-c959ac8723ff None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:43.012185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-df4b668f-a25b-46ad-8929-c959ac8723ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:43.043369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1204/2417] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:43.043369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b668f-a25b-46ad-8929-c959ac8723ff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.051416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b668f-a25b-46ad-8929-c959ac8723ff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.058927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78f2de-ab4e-492f-b329-f8faa35d062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.059758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78f2de-ab4e-492f-b329-f8faa35d062a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.060048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78f2de-ab4e-492f-b329-f8faa35d062a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.060441 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78f2de-ab4e-492f-b329-f8faa35d062a admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.061203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78f2de-ab4e-492f-b329-f8faa35d062a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:21:43.061602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78f2de-ab4e-492f-b329-f8faa35d062a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.062102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1214/2418] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:42 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:43.063339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78f2de-ab4e-492f-b329-f8faa35d062a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.063478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefabb52-2ca4-440a-bdc8-c172372ce2c8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.083357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1205/2419] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:21:43.084389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78f2de-ab4e-492f-b329-f8faa35d062a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.096551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78f2de-ab4e-492f-b329-f8faa35d062a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.153776 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefabb52-2ca4-440a-bdc8-c172372ce2c8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.158849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9032b25-1f8d-41cc-be1f-b16dc0d261be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.159773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9032b25-1f8d-41cc-be1f-b16dc0d261be admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.159905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9032b25-1f8d-41cc-be1f-b16dc0d261be admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.160126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9032b25-1f8d-41cc-be1f-b16dc0d261be admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.160823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9032b25-1f8d-41cc-be1f-b16dc0d261be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.162917 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9032b25-1f8d-41cc-be1f-b16dc0d261be admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.177005 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a322b-c799-4c7a-af4d-b79923a37e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.178101 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a322b-c799-4c7a-af4d-b79923a37e30 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.178439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a322b-c799-4c7a-af4d-b79923a37e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.178810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a322b-c799-4c7a-af4d-b79923a37e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.179723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a322b-c799-4c7a-af4d-b79923a37e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.181709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a322b-c799-4c7a-af4d-b79923a37e30 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.197246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1215/2420] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:43 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:43.203476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9032b25-1f8d-41cc-be1f-b16dc0d261be admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.219180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9032b25-1f8d-41cc-be1f-b16dc0d261be admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.224564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1206/2421] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:43 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:21:43.225923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360c4d6-5c12-41da-b26a-0eab49337af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.226599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360c4d6-5c12-41da-b26a-0eab49337af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.226893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360c4d6-5c12-41da-b26a-0eab49337af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.227362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360c4d6-5c12-41da-b26a-0eab49337af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.228032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360c4d6-5c12-41da-b26a-0eab49337af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.228832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a322b-c799-4c7a-af4d-b79923a37e30 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.229797 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360c4d6-5c12-41da-b26a-0eab49337af1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.247147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1216/2422] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:43.248888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360c4d6-5c12-41da-b26a-0eab49337af1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.263587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360c4d6-5c12-41da-b26a-0eab49337af1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.338106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a322b-c799-4c7a-af4d-b79923a37e30 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.346564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdaf73e-878b-4282-9c52-af9e8c5b0d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.348081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdaf73e-878b-4282-9c52-af9e8c5b0d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.348380 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdaf73e-878b-4282-9c52-af9e8c5b0d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.348716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdaf73e-878b-4282-9c52-af9e8c5b0d29 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.349477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdaf73e-878b-4282-9c52-af9e8c5b0d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.351603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdaf73e-878b-4282-9c52-af9e8c5b0d29 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.353156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdc97c-b2ce-4ae4-8d34-ef9f57eeec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.353837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdc97c-b2ce-4ae4-8d34-ef9f57eeec1e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.354126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdc97c-b2ce-4ae4-8d34-ef9f57eeec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.354408 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdc97c-b2ce-4ae4-8d34-ef9f57eeec1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.355147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cdc97c-b2ce-4ae4-8d34-ef9f57eeec1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.356961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cdc97c-b2ce-4ae4-8d34-ef9f57eeec1e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.370733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1217/2423] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:43.375999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdc97c-b2ce-4ae4-8d34-ef9f57eeec1e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.381207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1207/2424] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:43 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:43.385457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdaf73e-878b-4282-9c52-af9e8c5b0d29 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.394039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdc97c-b2ce-4ae4-8d34-ef9f57eeec1e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.396235 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdaf73e-878b-4282-9c52-af9e8c5b0d29 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.399769 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90745d-ea33-4609-aedb-971feb88ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.401191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90745d-ea33-4609-aedb-971feb88ea15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.401491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90745d-ea33-4609-aedb-971feb88ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.401769 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90745d-ea33-4609-aedb-971feb88ea15 admin admin] PATH_INFO: `/v3/projects/be45fd5ab38443589b9e79481a085a2e/users/4289c3ea663943df8f7673661e071bc5/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.402886 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54316885-982e-43e2-8139-09f4e2038fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.404031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54316885-982e-43e2-8139-09f4e2038fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.404436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54316885-982e-43e2-8139-09f4e2038fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.404821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54316885-982e-43e2-8139-09f4e2038fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.405682 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54316885-982e-43e2-8139-09f4e2038fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.408558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54316885-982e-43e2-8139-09f4e2038fd2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.424059 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1208/2425] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:43.425496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90745d-ea33-4609-aedb-971feb88ea15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be45fd5ab38443589b9e79481a085a2e, user_id=4289c3ea663943df8f7673661e071bc5, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.428557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90745d-ea33-4609-aedb-971feb88ea15 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.433267 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54316885-982e-43e2-8139-09f4e2038fd2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.447117 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54316885-982e-43e2-8139-09f4e2038fd2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.453965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1218/2426] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:43 2022] PUT /identity/v3/projects/be45fd5ab38443589b9e79481a085a2e/users/4289c3ea663943df8f7673661e071bc5/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:43.466805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5ff15-8932-4c8c-9630-910cd5e841ad None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.468041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5ff15-8932-4c8c-9630-910cd5e841ad None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.468340 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5ff15-8932-4c8c-9630-910cd5e841ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.496311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:43.506011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bbf5ff15-8932-4c8c-9630-910cd5e841ad None None] MFA Rules not processed for user `4289c3ea663943df8f7673661e071bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:43.508298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bbf5ff15-8932-4c8c-9630-910cd5e841ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:43.517450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bfc1f-cec2-4bfe-a39f-3a8636ce4829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.518165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bfc1f-cec2-4bfe-a39f-3a8636ce4829 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.518466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bfc1f-cec2-4bfe-a39f-3a8636ce4829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.518750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bfc1f-cec2-4bfe-a39f-3a8636ce4829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.520151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bfc1f-cec2-4bfe-a39f-3a8636ce4829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.521366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bfc1f-cec2-4bfe-a39f-3a8636ce4829 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.525330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:43.534847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1209/2427] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:43.539036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bfc1f-cec2-4bfe-a39f-3a8636ce4829 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.550458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bfc1f-cec2-4bfe-a39f-3a8636ce4829 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.555547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f88b2-fb25-418b-ae03-574a8ef51d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.556552 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f88b2-fb25-418b-ae03-574a8ef51d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.556856 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f88b2-fb25-418b-ae03-574a8ef51d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.557187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f88b2-fb25-418b-ae03-574a8ef51d6d admin admin] PATH_INFO: `/v3/projects/e821ac4b84334725841d778335991c1f/users/cc04b2f36a6e4d7db1a231c7ff3525bc/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.575683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f88b2-fb25-418b-ae03-574a8ef51d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e821ac4b84334725841d778335991c1f, user_id=cc04b2f36a6e4d7db1a231c7ff3525bc, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.578374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f88b2-fb25-418b-ae03-574a8ef51d6d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.594662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1210/2428] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:43 2022] PUT /identity/v3/projects/e821ac4b84334725841d778335991c1f/users/cc04b2f36a6e4d7db1a231c7ff3525bc/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:43.605679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98739b4-6311-40fe-998d-5e1018af6376 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.605978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98739b4-6311-40fe-998d-5e1018af6376 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.606255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98739b4-6311-40fe-998d-5e1018af6376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.608966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1219/2429] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:43.615152 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5ff15-8932-4c8c-9630-910cd5e841ad None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.629355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5ff15-8932-4c8c-9630-910cd5e841ad None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.641381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e98739b4-6311-40fe-998d-5e1018af6376 None None] MFA Rules not processed for user `cc04b2f36a6e4d7db1a231c7ff3525bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:43.642899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e98739b4-6311-40fe-998d-5e1018af6376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:43.715231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67852d-efbd-4a88-858e-53759957d1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.716638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67852d-efbd-4a88-858e-53759957d1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.717086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67852d-efbd-4a88-858e-53759957d1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.717548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67852d-efbd-4a88-858e-53759957d1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.719536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67852d-efbd-4a88-858e-53759957d1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.725011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67852d-efbd-4a88-858e-53759957d1d5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.744070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1211/2430] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:43.756604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98739b4-6311-40fe-998d-5e1018af6376 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.758428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e98739b4-6311-40fe-998d-5e1018af6376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:43.759398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1220/2431] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:43 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:21:43.763638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67852d-efbd-4a88-858e-53759957d1d5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.838959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98739b4-6311-40fe-998d-5e1018af6376 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.857414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67852d-efbd-4a88-858e-53759957d1d5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.867439 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56600946-7136-4628-896e-9d0978be2f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.868518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56600946-7136-4628-896e-9d0978be2f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.868992 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56600946-7136-4628-896e-9d0978be2f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.869469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56600946-7136-4628-896e-9d0978be2f53 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.870738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56600946-7136-4628-896e-9d0978be2f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.874248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56600946-7136-4628-896e-9d0978be2f53 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.903192 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1221/2432] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:43 2022] POST /identity/v3/users => generated 300 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:43.908276 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56600946-7136-4628-896e-9d0978be2f53 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.917360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbd0e3-598d-480e-a25c-20925699997a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.917958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbd0e3-598d-480e-a25c-20925699997a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.918245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbd0e3-598d-480e-a25c-20925699997a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.919048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbd0e3-598d-480e-a25c-20925699997a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.919175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56600946-7136-4628-896e-9d0978be2f53 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.923972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86c63a-ae76-41fa-b1d4-5fa609d6873d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.924683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86c63a-ae76-41fa-b1d4-5fa609d6873d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.924970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86c63a-ae76-41fa-b1d4-5fa609d6873d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.925246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86c63a-ae76-41fa-b1d4-5fa609d6873d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.926195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86c63a-ae76-41fa-b1d4-5fa609d6873d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.927827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86c63a-ae76-41fa-b1d4-5fa609d6873d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.933028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:43.939770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1222/2433] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:43.944773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86c63a-ae76-41fa-b1d4-5fa609d6873d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:43.956586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86c63a-ae76-41fa-b1d4-5fa609d6873d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:43.963169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353f377-8474-4f22-82ec-68e192c41734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:43.964209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353f377-8474-4f22-82ec-68e192c41734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:43.964609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353f377-8474-4f22-82ec-68e192c41734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:43.964959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353f377-8474-4f22-82ec-68e192c41734 admin admin] PATH_INFO: `/v3/projects/162de56ac15b411dae405d0b8317d83b/users/86769ce2289c4c93a33de510bc19da59/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:43.982167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353f377-8474-4f22-82ec-68e192c41734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162de56ac15b411dae405d0b8317d83b, user_id=86769ce2289c4c93a33de510bc19da59, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:43.984980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353f377-8474-4f22-82ec-68e192c41734 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:43.988940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:44.002825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1223/2434] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:43 2022] PUT /identity/v3/projects/162de56ac15b411dae405d0b8317d83b/users/86769ce2289c4c93a33de510bc19da59/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:44.003045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbd0e3-598d-480e-a25c-20925699997a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:44.004314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbd0e3-598d-480e-a25c-20925699997a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:44.006614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353f377-8474-4f22-82ec-68e192c41734 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:44.014103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1212/2435] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:21:44.088365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353f377-8474-4f22-82ec-68e192c41734 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:44.097270 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eed5c2-7278-4fb4-847a-a600d5adb81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:44.097995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eed5c2-7278-4fb4-847a-a600d5adb81a admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:44.098272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eed5c2-7278-4fb4-847a-a600d5adb81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:44.098556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eed5c2-7278-4fb4-847a-a600d5adb81a admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:44.099321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eed5c2-7278-4fb4-847a-a600d5adb81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:44.101230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eed5c2-7278-4fb4-847a-a600d5adb81a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:44.135116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1224/2436] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:44.135511 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbd0e3-598d-480e-a25c-20925699997a service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:44.154028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbd0e3-598d-480e-a25c-20925699997a service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:44.162078 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7d4ec-11c1-4647-83b7-48af11a01b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:44.163182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7d4ec-11c1-4647-83b7-48af11a01b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:44.163708 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7d4ec-11c1-4647-83b7-48af11a01b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:44.164826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7d4ec-11c1-4647-83b7-48af11a01b0c admin admin] PATH_INFO: `/v3/projects/162de56ac15b411dae405d0b8317d83b/users/86769ce2289c4c93a33de510bc19da59/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:44.168260 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7d4ec-11c1-4647-83b7-48af11a01b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=162de56ac15b411dae405d0b8317d83b, user_id=86769ce2289c4c93a33de510bc19da59, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:44.173105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7d4ec-11c1-4647-83b7-48af11a01b0c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:44.203464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489a6fb-0ee1-40a3-967b-9bcc8fb1d566 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:44.205308 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489a6fb-0ee1-40a3-967b-9bcc8fb1d566 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:44.205752 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489a6fb-0ee1-40a3-967b-9bcc8fb1d566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:44.207829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1213/2437] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:44 2022] PUT /identity/v3/projects/162de56ac15b411dae405d0b8317d83b/users/86769ce2289c4c93a33de510bc19da59/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:44.239738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0489a6fb-0ee1-40a3-967b-9bcc8fb1d566 None None] MFA Rules not processed for user `86769ce2289c4c93a33de510bc19da59`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:44.241151 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0489a6fb-0ee1-40a3-967b-9bcc8fb1d566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:44.331728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1225/2438] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:44.352143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7d4ec-11c1-4647-83b7-48af11a01b0c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:44.359962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7df7d4ec-11c1-4647-83b7-48af11a01b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:44.432472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7d4ec-11c1-4647-83b7-48af11a01b0c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:44.436724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151d460-f4d3-4fce-a3e9-f6328a1cc464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:44.437724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151d460-f4d3-4fce-a3e9-f6328a1cc464 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:44.438013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151d460-f4d3-4fce-a3e9-f6328a1cc464 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:44.438290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151d460-f4d3-4fce-a3e9-f6328a1cc464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:44.451887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151d460-f4d3-4fce-a3e9-f6328a1cc464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:44.453175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151d460-f4d3-4fce-a3e9-f6328a1cc464 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:44.466098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1214/2439] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:44.545208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489a6fb-0ee1-40a3-967b-9bcc8fb1d566 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:44.620868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489a6fb-0ee1-40a3-967b-9bcc8fb1d566 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:44.625812 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:44.627458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:44.627802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:44.628089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] PATH_INFO: `/v3/users/4289c3ea663943df8f7673661e071bc5` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:44.629470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4289c3ea663943df8f7673661e071bc5)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:44.631480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:44.655658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:44.698645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4289c3ea663943df8f7673661e071bc5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:44.699510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4289c3ea663943df8f7673661e071bc5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:44.708191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4289c3ea663943df8f7673661e071bc5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:44.721028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4289c3ea663943df8f7673661e071bc5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:44.728051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4289c3ea663943df8f7673661e071bc5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:44.734649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1226/2440] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:44 2022] DELETE /identity/v3/users/4289c3ea663943df8f7673661e071bc5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:44.740452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e25dd9-9846-4296-9a3c-2ebacadbc5f0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:44.740745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e25dd9-9846-4296-9a3c-2ebacadbc5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:44.741079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e25dd9-9846-4296-9a3c-2ebacadbc5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:44.761135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-31e25dd9-9846-4296-9a3c-2ebacadbc5f0 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:44.762366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-31e25dd9-9846-4296-9a3c-2ebacadbc5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:44.824007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1215/2441] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:44.833835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:44.835322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:44.894512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867bdb9-a07d-4e05-9f3c-e54edc2f2989 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:44.898350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c5c0c-9ac1-486c-9407-6aa380e1ed8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:44.899039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c5c0c-9ac1-486c-9407-6aa380e1ed8e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:44.899362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c5c0c-9ac1-486c-9407-6aa380e1ed8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:44.899645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c5c0c-9ac1-486c-9407-6aa380e1ed8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:44.910119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c5c0c-9ac1-486c-9407-6aa380e1ed8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:44.911590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c5c0c-9ac1-486c-9407-6aa380e1ed8e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:44.921432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1227/2442] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:45.119592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bb35d-4994-4f31-a272-5e8da4753287 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:45.120106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bb35d-4994-4f31-a272-5e8da4753287 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:45.120207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bb35d-4994-4f31-a272-5e8da4753287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:45.139304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-339bb35d-4994-4f31-a272-5e8da4753287 None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:45.140541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-339bb35d-4994-4f31-a272-5e8da4753287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:45.185281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:45.202994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c5c0c-9ac1-486c-9407-6aa380e1ed8e service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:45.203941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1216/2443] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:21:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:45.204505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4f6c5c0c-9ac1-486c-9407-6aa380e1ed8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:45.218905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c5c0c-9ac1-486c-9407-6aa380e1ed8e service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:45.219777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bb35d-4994-4f31-a272-5e8da4753287 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:45.221930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-339bb35d-4994-4f31-a272-5e8da4753287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:45.224433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:45.225436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:45.225823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:45.226201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] PATH_INFO: `/v3/projects/be45fd5ab38443589b9e79481a085a2e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:45.227580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be45fd5ab38443589b9e79481a085a2e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:45.229838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:45.278911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be45fd5ab38443589b9e79481a085a2e', 'request_id': 'req-6431b74e-1cbc-4416-b191-903ce0d9ff45'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.285328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be45fd5ab38443589b9e79481a085a2e', 'request_id': 'req-6431b74e-1cbc-4416-b191-903ce0d9ff45'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.292306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be45fd5ab38443589b9e79481a085a2e', 'request_id': 'req-6431b74e-1cbc-4416-b191-903ce0d9ff45'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.297057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bb35d-4994-4f31-a272-5e8da4753287 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:45.311154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] The token cache is being invalidate because project be45fd5ab38443589b9e79481a085a2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:45.312058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.313003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.316403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1228/2444] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:45 2022] DELETE /identity/v3/projects/be45fd5ab38443589b9e79481a085a2e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:45.320482 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:45.322603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:45.355521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:45.361792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aed0b9-1e4a-4530-b574-3998bc31be01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:45.362463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aed0b9-1e4a-4530-b574-3998bc31be01 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:45.362754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aed0b9-1e4a-4530-b574-3998bc31be01 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:45.363036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aed0b9-1e4a-4530-b574-3998bc31be01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:45.365654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-29aed0b9-1e4a-4530-b574-3998bc31be01 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:45.399546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431b74e-1cbc-4416-b191-903ce0d9ff45 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:45.404330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:45.405224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:45.405521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:45.405798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] PATH_INFO: `/v3/users/86769ce2289c4c93a33de510bc19da59` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:45.407051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86769ce2289c4c93a33de510bc19da59)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:45.409588 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:45.439188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aed0b9-1e4a-4530-b574-3998bc31be01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:45.440851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aed0b9-1e4a-4530-b574-3998bc31be01 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:45.453074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1217/2445] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:21:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:21:45.484985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86769ce2289c4c93a33de510bc19da59'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.485977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86769ce2289c4c93a33de510bc19da59'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.497084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86769ce2289c4c93a33de510bc19da59'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.510994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86769ce2289c4c93a33de510bc19da59'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.529822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0847dfaa-dd04-4f04-b10c-62eb1bbcd783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86769ce2289c4c93a33de510bc19da59'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:45.559720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1229/2446] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:45 2022] DELETE /identity/v3/users/86769ce2289c4c93a33de510bc19da59 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:46.112539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aed0b9-1e4a-4530-b574-3998bc31be01 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:46.137557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aed0b9-1e4a-4530-b574-3998bc31be01 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:46.210954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:46.254475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:46.255576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.255867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.256148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] PATH_INFO: `/v3/projects/162de56ac15b411dae405d0b8317d83b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.257398 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=162de56ac15b411dae405d0b8317d83b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:46.260660 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:46.357734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '162de56ac15b411dae405d0b8317d83b', 'request_id': 'req-063aab43-b83a-45d4-8824-31ba9d19fcd8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:46.368636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '162de56ac15b411dae405d0b8317d83b', 'request_id': 'req-063aab43-b83a-45d4-8824-31ba9d19fcd8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:46.375834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '162de56ac15b411dae405d0b8317d83b', 'request_id': 'req-063aab43-b83a-45d4-8824-31ba9d19fcd8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:46.394284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] The token cache is being invalidate because project 162de56ac15b411dae405d0b8317d83b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:46.394672 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:46.395643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-063aab43-b83a-45d4-8824-31ba9d19fcd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:46.397330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1218/2447] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:46 2022] DELETE /identity/v3/projects/162de56ac15b411dae405d0b8317d83b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:46.402649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1750086-ba87-471a-adb2-2fd13da3595d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.403193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1750086-ba87-471a-adb2-2fd13da3595d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.403193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1750086-ba87-471a-adb2-2fd13da3595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.424704 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b1750086-ba87-471a-adb2-2fd13da3595d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:46.426271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b1750086-ba87-471a-adb2-2fd13da3595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:46.521250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1230/2448] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:46.529532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3dc44-b5cc-472c-a760-543f15d211cf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.529965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3dc44-b5cc-472c-a760-543f15d211cf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.530352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3dc44-b5cc-472c-a760-543f15d211cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.561038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a4d3dc44-b5cc-472c-a760-543f15d211cf None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:46.562533 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a4d3dc44-b5cc-472c-a760-543f15d211cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:46.575314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1219/2449] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:46.576891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1750086-ba87-471a-adb2-2fd13da3595d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:46.602572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1750086-ba87-471a-adb2-2fd13da3595d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:46.608672 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100b7c5-fc06-469d-8b3e-81fd6545d5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:46.609419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100b7c5-fc06-469d-8b3e-81fd6545d5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.609713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100b7c5-fc06-469d-8b3e-81fd6545d5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.609994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100b7c5-fc06-469d-8b3e-81fd6545d5c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.611332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100b7c5-fc06-469d-8b3e-81fd6545d5c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:21:46.612155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100b7c5-fc06-469d-8b3e-81fd6545d5c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:46.613840 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100b7c5-fc06-469d-8b3e-81fd6545d5c5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:46.636444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1231/2450] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:21:46.637612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3dc44-b5cc-472c-a760-543f15d211cf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:46.648667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3dc44-b5cc-472c-a760-543f15d211cf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:46.657989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997e851-c770-40a6-a47a-053ab1e5773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:46.658719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997e851-c770-40a6-a47a-053ab1e5773d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.659014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997e851-c770-40a6-a47a-053ab1e5773d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.659306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997e851-c770-40a6-a47a-053ab1e5773d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.660159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997e851-c770-40a6-a47a-053ab1e5773d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:46.662347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997e851-c770-40a6-a47a-053ab1e5773d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:46.697693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1220/2451] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:46 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:46.701534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100b7c5-fc06-469d-8b3e-81fd6545d5c5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:46.712590 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100b7c5-fc06-469d-8b3e-81fd6545d5c5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:46.744703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:46.780568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dccd88-6487-4873-9fad-778bcd5eabac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:46.781244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dccd88-6487-4873-9fad-778bcd5eabac admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.781540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dccd88-6487-4873-9fad-778bcd5eabac admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.781822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dccd88-6487-4873-9fad-778bcd5eabac admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.783096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dccd88-6487-4873-9fad-778bcd5eabac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:46.785779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dccd88-6487-4873-9fad-778bcd5eabac admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:46.807991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1232/2452] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:46 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:46.812626 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997e851-c770-40a6-a47a-053ab1e5773d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:46.823529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997e851-c770-40a6-a47a-053ab1e5773d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:46.828423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643df37-830e-495c-a71b-6385cd031854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:46.829104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643df37-830e-495c-a71b-6385cd031854 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.829911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643df37-830e-495c-a71b-6385cd031854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.830182 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643df37-830e-495c-a71b-6385cd031854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.830927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643df37-830e-495c-a71b-6385cd031854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:46.832583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643df37-830e-495c-a71b-6385cd031854 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:46.846132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1221/2453] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:46.850239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dccd88-6487-4873-9fad-778bcd5eabac admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:46.860681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dccd88-6487-4873-9fad-778bcd5eabac admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:46.865854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a13102-e1fc-4bec-9961-a0d92df2f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:46.866541 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a13102-e1fc-4bec-9961-a0d92df2f9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.866826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a13102-e1fc-4bec-9961-a0d92df2f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.867105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a13102-e1fc-4bec-9961-a0d92df2f9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.867886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a13102-e1fc-4bec-9961-a0d92df2f9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:46.869518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a13102-e1fc-4bec-9961-a0d92df2f9a7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:46.880545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1233/2454] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:46.884618 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643df37-830e-495c-a71b-6385cd031854 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:46.895748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643df37-830e-495c-a71b-6385cd031854 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:46.900739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf36f85-6c9c-43dd-917e-94cd0b03ead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:46.901658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf36f85-6c9c-43dd-917e-94cd0b03ead5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.901940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf36f85-6c9c-43dd-917e-94cd0b03ead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.902524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf36f85-6c9c-43dd-917e-94cd0b03ead5 admin admin] PATH_INFO: `/v3/projects/e5f2d9d01a204b5da03a2c7e1eee80e7/users/f31a6a36e475420b82cac82b411e67ba/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.920434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf36f85-6c9c-43dd-917e-94cd0b03ead5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f2d9d01a204b5da03a2c7e1eee80e7, user_id=f31a6a36e475420b82cac82b411e67ba, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:46.923308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf36f85-6c9c-43dd-917e-94cd0b03ead5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:46.939990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1222/2455] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:46 2022] PUT /identity/v3/projects/e5f2d9d01a204b5da03a2c7e1eee80e7/users/f31a6a36e475420b82cac82b411e67ba/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:46.951409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d1a6-e9cc-4dd2-8c83-d42fe3358caa None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:46.951628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d1a6-e9cc-4dd2-8c83-d42fe3358caa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:46.951867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d1a6-e9cc-4dd2-8c83-d42fe3358caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:46.982248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3186d1a6-e9cc-4dd2-8c83-d42fe3358caa None None] MFA Rules not processed for user `f31a6a36e475420b82cac82b411e67ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:46.983604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3186d1a6-e9cc-4dd2-8c83-d42fe3358caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:47.074605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1234/2456] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:47.076010 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf36f85-6c9c-43dd-917e-94cd0b03ead5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:47.150108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf36f85-6c9c-43dd-917e-94cd0b03ead5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:47.155871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846ae21-9f6b-426b-9502-12b068d4dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:47.156561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846ae21-9f6b-426b-9502-12b068d4dea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:47.156976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846ae21-9f6b-426b-9502-12b068d4dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:47.157270 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846ae21-9f6b-426b-9502-12b068d4dea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:47.158226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846ae21-9f6b-426b-9502-12b068d4dea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:47.163700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846ae21-9f6b-426b-9502-12b068d4dea8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:47.167669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:47.216816 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d1a6-e9cc-4dd2-8c83-d42fe3358caa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:47.219187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1223/2457] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:47 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:47.232268 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d1a6-e9cc-4dd2-8c83-d42fe3358caa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:47.344932 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75e6c9-67a5-4f94-9544-4ab121ac82cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:47.345821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e6c9-67a5-4f94-9544-4ab121ac82cf admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:47.346292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e6c9-67a5-4f94-9544-4ab121ac82cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:47.346759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e6c9-67a5-4f94-9544-4ab121ac82cf admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:47.347993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75e6c9-67a5-4f94-9544-4ab121ac82cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:47.350592 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75e6c9-67a5-4f94-9544-4ab121ac82cf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:47.389212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1235/2458] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:47 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:21:47.391359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846ae21-9f6b-426b-9502-12b068d4dea8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:47.407623 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846ae21-9f6b-426b-9502-12b068d4dea8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:47.414793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b3d44-a6b4-4e3c-81e2-8c70488c1d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:47.415519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b3d44-a6b4-4e3c-81e2-8c70488c1d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:47.415815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b3d44-a6b4-4e3c-81e2-8c70488c1d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:47.416097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b3d44-a6b4-4e3c-81e2-8c70488c1d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:47.416834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b3d44-a6b4-4e3c-81e2-8c70488c1d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:47.418474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b3d44-a6b4-4e3c-81e2-8c70488c1d68 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:47.429912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1224/2459] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:47.434357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75e6c9-67a5-4f94-9544-4ab121ac82cf admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:47.458180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75e6c9-67a5-4f94-9544-4ab121ac82cf admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:47.462997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e87be-e7a5-4533-a1db-995603fa74c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:47.463721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e87be-e7a5-4533-a1db-995603fa74c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:47.464297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e87be-e7a5-4533-a1db-995603fa74c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:47.464589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e87be-e7a5-4533-a1db-995603fa74c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:47.466086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e87be-e7a5-4533-a1db-995603fa74c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:47.467759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e87be-e7a5-4533-a1db-995603fa74c7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:47.480314 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1236/2460] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:47.484895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b3d44-a6b4-4e3c-81e2-8c70488c1d68 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:47.496223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b3d44-a6b4-4e3c-81e2-8c70488c1d68 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:47.502906 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790d29b-67b5-4561-8123-77182838b669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:47.503962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790d29b-67b5-4561-8123-77182838b669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:47.504329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790d29b-67b5-4561-8123-77182838b669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:47.504671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790d29b-67b5-4561-8123-77182838b669 admin admin] PATH_INFO: `/v3/projects/14a902f69a974cb9bc4c4bf95830fde5/users/c2ee3d2286224fd9aa1d430ad5a9bbe3/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:47.527167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790d29b-67b5-4561-8123-77182838b669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a902f69a974cb9bc4c4bf95830fde5, user_id=c2ee3d2286224fd9aa1d430ad5a9bbe3, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:47.530932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790d29b-67b5-4561-8123-77182838b669 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:47.548996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1225/2461] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:47 2022] PUT /identity/v3/projects/14a902f69a974cb9bc4c4bf95830fde5/users/c2ee3d2286224fd9aa1d430ad5a9bbe3/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:47.559571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb0d6c-bd59-48bf-8e51-7b46603954b3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:47.559861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb0d6c-bd59-48bf-8e51-7b46603954b3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:47.560140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb0d6c-bd59-48bf-8e51-7b46603954b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:47.606306 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-59cb0d6c-bd59-48bf-8e51-7b46603954b3 None None] MFA Rules not processed for user `c2ee3d2286224fd9aa1d430ad5a9bbe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:47.608284 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-59cb0d6c-bd59-48bf-8e51-7b46603954b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:47.613705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:47.755622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1237/2462] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:47.760635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790d29b-67b5-4561-8123-77182838b669 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:47.863827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790d29b-67b5-4561-8123-77182838b669 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:47.869183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e82b6-2945-4e63-8ca2-c354d12f06af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:47.869868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e82b6-2945-4e63-8ca2-c354d12f06af admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:47.870162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e82b6-2945-4e63-8ca2-c354d12f06af admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:47.870451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e82b6-2945-4e63-8ca2-c354d12f06af admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:47.871232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e82b6-2945-4e63-8ca2-c354d12f06af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:47.873323 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e82b6-2945-4e63-8ca2-c354d12f06af admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:47.904967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1226/2463] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:47 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:47.906015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb0d6c-bd59-48bf-8e51-7b46603954b3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:47.931216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb0d6c-bd59-48bf-8e51-7b46603954b3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:48.038979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e885b06-46f3-4dbc-a31b-cd6d575448c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:48.039825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e885b06-46f3-4dbc-a31b-cd6d575448c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:48.040480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e885b06-46f3-4dbc-a31b-cd6d575448c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:48.040480 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e885b06-46f3-4dbc-a31b-cd6d575448c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:48.041208 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e885b06-46f3-4dbc-a31b-cd6d575448c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:48.043487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e885b06-46f3-4dbc-a31b-cd6d575448c1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:48.071137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1238/2464] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:47 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:48.075225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e82b6-2945-4e63-8ca2-c354d12f06af admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:48.086348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e82b6-2945-4e63-8ca2-c354d12f06af admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:48.090872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01ed43-1186-448c-b1ea-ed14b6e50759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:48.091585 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01ed43-1186-448c-b1ea-ed14b6e50759 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:48.091880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01ed43-1186-448c-b1ea-ed14b6e50759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:48.092279 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01ed43-1186-448c-b1ea-ed14b6e50759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:48.093028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01ed43-1186-448c-b1ea-ed14b6e50759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:48.094794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01ed43-1186-448c-b1ea-ed14b6e50759 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:48.107686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1227/2465] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:48.112172 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e885b06-46f3-4dbc-a31b-cd6d575448c1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:48.125328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e885b06-46f3-4dbc-a31b-cd6d575448c1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:48.131324 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc8f32-8d12-48ef-9b7e-7403fdc3386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:48.132282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc8f32-8d12-48ef-9b7e-7403fdc3386f admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:48.132575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc8f32-8d12-48ef-9b7e-7403fdc3386f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:48.132881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc8f32-8d12-48ef-9b7e-7403fdc3386f admin admin] PATH_INFO: `/v3/projects/ad6866112de94a239115ee9b48304b46/users/83ce9ee164b64753936ffd1ab20c5731/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:48.151625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc8f32-8d12-48ef-9b7e-7403fdc3386f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6866112de94a239115ee9b48304b46, user_id=83ce9ee164b64753936ffd1ab20c5731, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:48.154289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc8f32-8d12-48ef-9b7e-7403fdc3386f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:48.170046 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1239/2466] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:48 2022] PUT /identity/v3/projects/ad6866112de94a239115ee9b48304b46/users/83ce9ee164b64753936ffd1ab20c5731/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:48.174107 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01ed43-1186-448c-b1ea-ed14b6e50759 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:48.185103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01ed43-1186-448c-b1ea-ed14b6e50759 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:48.252268 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c300a-aea0-4373-ad7b-8a7552badaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:48.252996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c300a-aea0-4373-ad7b-8a7552badaad admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:48.253288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c300a-aea0-4373-ad7b-8a7552badaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:48.253563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c300a-aea0-4373-ad7b-8a7552badaad admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:48.254298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c300a-aea0-4373-ad7b-8a7552badaad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:48.256852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c300a-aea0-4373-ad7b-8a7552badaad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:48.268611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1228/2467] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:48.272781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc8f32-8d12-48ef-9b7e-7403fdc3386f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:48.292551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc8f32-8d12-48ef-9b7e-7403fdc3386f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:48.306468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ae1e4-0e67-4c73-8f7b-d0d6f4fe0139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:48.307410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ae1e4-0e67-4c73-8f7b-d0d6f4fe0139 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:48.307700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ae1e4-0e67-4c73-8f7b-d0d6f4fe0139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:48.308071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ae1e4-0e67-4c73-8f7b-d0d6f4fe0139 admin admin] PATH_INFO: `/v3/projects/ad6866112de94a239115ee9b48304b46/users/83ce9ee164b64753936ffd1ab20c5731/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:48.310684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ae1e4-0e67-4c73-8f7b-d0d6f4fe0139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6866112de94a239115ee9b48304b46, user_id=83ce9ee164b64753936ffd1ab20c5731, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:48.313864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ae1e4-0e67-4c73-8f7b-d0d6f4fe0139 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:48.325825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1240/2468] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:48 2022] PUT /identity/v3/projects/ad6866112de94a239115ee9b48304b46/users/83ce9ee164b64753936ffd1ab20c5731/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:48.337155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c044-3c12-403a-a8d4-dbbbd4053a18 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:48.337460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c044-3c12-403a-a8d4-dbbbd4053a18 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:48.337749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c044-3c12-403a-a8d4-dbbbd4053a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:48.368628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2f21c044-3c12-403a-a8d4-dbbbd4053a18 None None] MFA Rules not processed for user `83ce9ee164b64753936ffd1ab20c5731`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:48.369872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f21c044-3c12-403a-a8d4-dbbbd4053a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:48.385084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:48.470807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1229/2469] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:48.482759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ae1e4-0e67-4c73-8f7b-d0d6f4fe0139 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:48.484604 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-082ae1e4-0e67-4c73-8f7b-d0d6f4fe0139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:48.551798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ae1e4-0e67-4c73-8f7b-d0d6f4fe0139 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:48.555586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ace7b1-5bd2-4984-8c20-375072c531c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:48.556247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ace7b1-5bd2-4984-8c20-375072c531c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:48.556531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ace7b1-5bd2-4984-8c20-375072c531c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:48.556806 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ace7b1-5bd2-4984-8c20-375072c531c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:48.567186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ace7b1-5bd2-4984-8c20-375072c531c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:48.568721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ace7b1-5bd2-4984-8c20-375072c531c7 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:48.574829 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:48.580728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1241/2470] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:49.427813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c044-3c12-403a-a8d4-dbbbd4053a18 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:49.440103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c044-3c12-403a-a8d4-dbbbd4053a18 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:49.444761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cde42-c04e-4519-a615-7579f5fa2486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:49.445658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cde42-c04e-4519-a615-7579f5fa2486 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:49.446114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cde42-c04e-4519-a615-7579f5fa2486 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:49.446549 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cde42-c04e-4519-a615-7579f5fa2486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:49.580620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cde42-c04e-4519-a615-7579f5fa2486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:49.582936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cde42-c04e-4519-a615-7579f5fa2486 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:49.601932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1230/2471] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:53.464243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ace7b1-5bd2-4984-8c20-375072c531c7 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:53.465783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f7ace7b1-5bd2-4984-8c20-375072c531c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:53.607374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ace7b1-5bd2-4984-8c20-375072c531c7 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:53.612834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:53.613725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:53.614012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:53.614556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] PATH_INFO: `/v3/users/cc04b2f36a6e4d7db1a231c7ff3525bc` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:53.615803 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc04b2f36a6e4d7db1a231c7ff3525bc)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:53.617758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:53.665661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc04b2f36a6e4d7db1a231c7ff3525bc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:53.666160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc04b2f36a6e4d7db1a231c7ff3525bc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:53.677021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc04b2f36a6e4d7db1a231c7ff3525bc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:53.691494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc04b2f36a6e4d7db1a231c7ff3525bc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:53.698481 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc04b2f36a6e4d7db1a231c7ff3525bc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:53.704614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1242/2472] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:53 2022] DELETE /identity/v3/users/cc04b2f36a6e4d7db1a231c7ff3525bc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:53.709130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add455b5-1890-44d0-b234-1ada6c61383d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:53.709630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add455b5-1890-44d0-b234-1ada6c61383d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:53.709737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add455b5-1890-44d0-b234-1ada6c61383d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:53.744197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-add455b5-1890-44d0-b234-1ada6c61383d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:53.745519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-add455b5-1890-44d0-b234-1ada6c61383d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:53.812118 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1231/2473] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:53.826629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:53.828184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:53.850502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:53.910281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e5e84-e81c-4b8a-a6f3-c3b7a90830d6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:53.913931 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b3b01-d71a-4214-934e-683f89cfb8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:53.914617 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b3b01-d71a-4214-934e-683f89cfb8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:53.914868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b3b01-d71a-4214-934e-683f89cfb8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:53.915153 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b3b01-d71a-4214-934e-683f89cfb8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:53.931759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b3b01-d71a-4214-934e-683f89cfb8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:53.933209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b3b01-d71a-4214-934e-683f89cfb8d8 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:53.938727 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add455b5-1890-44d0-b234-1ada6c61383d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:53.940190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-add455b5-1890-44d0-b234-1ada6c61383d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:53.943766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1243/2474] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:53.983330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:54.006915 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add455b5-1890-44d0-b234-1ada6c61383d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:54.011640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a125988-0a30-4f45-8d95-1f20d47e36f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:54.012460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a125988-0a30-4f45-8d95-1f20d47e36f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.012823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a125988-0a30-4f45-8d95-1f20d47e36f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.013326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a125988-0a30-4f45-8d95-1f20d47e36f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:54.015909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6a125988-0a30-4f45-8d95-1f20d47e36f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:54.093822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a125988-0a30-4f45-8d95-1f20d47e36f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:54.095879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a125988-0a30-4f45-8d95-1f20d47e36f3 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:54.105986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1232/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:21:54.293544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b3b01-d71a-4214-934e-683f89cfb8d8 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:54.295184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4e1b3b01-d71a-4214-934e-683f89cfb8d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:54.314052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b3b01-d71a-4214-934e-683f89cfb8d8 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:54.321691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:54.322895 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.323364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.323601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] PATH_INFO: `/v3/projects/e821ac4b84334725841d778335991c1f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:54.325010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e821ac4b84334725841d778335991c1f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:54.327599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:54.393424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e821ac4b84334725841d778335991c1f', 'request_id': 'req-3b199be4-215a-42fd-ad60-30f74434925e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:54.401710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e821ac4b84334725841d778335991c1f', 'request_id': 'req-3b199be4-215a-42fd-ad60-30f74434925e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:54.412796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e821ac4b84334725841d778335991c1f', 'request_id': 'req-3b199be4-215a-42fd-ad60-30f74434925e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:54.441522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] The token cache is being invalidate because project e821ac4b84334725841d778335991c1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:54.441825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:54.443201 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3b199be4-215a-42fd-ad60-30f74434925e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:54.446053 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1244/2476] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:54 2022] DELETE /identity/v3/projects/e821ac4b84334725841d778335991c1f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:54.458781 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccb82e-e990-4d5d-8946-f7a9bbdea72a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.459069 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccb82e-e990-4d5d-8946-f7a9bbdea72a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.459370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccb82e-e990-4d5d-8946-f7a9bbdea72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:54.524896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-41ccb82e-e990-4d5d-8946-f7a9bbdea72a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:54.526446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-41ccb82e-e990-4d5d-8946-f7a9bbdea72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:54.617614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1233/2477] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:54.625279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f52273-1bb8-468e-aef1-000e1217e99d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.625827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f52273-1bb8-468e-aef1-000e1217e99d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.625942 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f52273-1bb8-468e-aef1-000e1217e99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:54.652877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-97f52273-1bb8-468e-aef1-000e1217e99d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:54.654144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-97f52273-1bb8-468e-aef1-000e1217e99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:54.663209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1245/2478] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:54.667800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccb82e-e990-4d5d-8946-f7a9bbdea72a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:54.677990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccb82e-e990-4d5d-8946-f7a9bbdea72a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:54.684315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f5a25-3f43-4ce0-8131-551c69ed301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:54.684315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f5a25-3f43-4ce0-8131-551c69ed301a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.687048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f5a25-3f43-4ce0-8131-551c69ed301a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.687048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f5a25-3f43-4ce0-8131-551c69ed301a admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:54.687048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f5a25-3f43-4ce0-8131-551c69ed301a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:21:54.687048 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f5a25-3f43-4ce0-8131-551c69ed301a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:54.688995 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f5a25-3f43-4ce0-8131-551c69ed301a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:54.700608 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1234/2479] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:21:54.705622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f52273-1bb8-468e-aef1-000e1217e99d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:54.715189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f52273-1bb8-468e-aef1-000e1217e99d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:54.719966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedfc42-7841-42a9-aaec-825708e798bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:54.720824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedfc42-7841-42a9-aaec-825708e798bc admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.721114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedfc42-7841-42a9-aaec-825708e798bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.721397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedfc42-7841-42a9-aaec-825708e798bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:54.722112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedfc42-7841-42a9-aaec-825708e798bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:54.724085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedfc42-7841-42a9-aaec-825708e798bc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:54.759609 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1246/2480] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:54 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:54.762093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f5a25-3f43-4ce0-8131-551c69ed301a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:54.773162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f5a25-3f43-4ce0-8131-551c69ed301a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:54.836204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:54.845759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04027a8c-f321-41bc-b82d-58f1860bad61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:54.846458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04027a8c-f321-41bc-b82d-58f1860bad61 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.846756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04027a8c-f321-41bc-b82d-58f1860bad61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.847037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04027a8c-f321-41bc-b82d-58f1860bad61 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:54.847844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04027a8c-f321-41bc-b82d-58f1860bad61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:54.850354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04027a8c-f321-41bc-b82d-58f1860bad61 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:54.886925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1235/2481] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:54 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:54.893767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedfc42-7841-42a9-aaec-825708e798bc admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:54.906857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedfc42-7841-42a9-aaec-825708e798bc admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:54.912155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c7fd6-fb4b-402d-88b3-09fb3e4a6ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:54.912836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c7fd6-fb4b-402d-88b3-09fb3e4a6ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.913120 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c7fd6-fb4b-402d-88b3-09fb3e4a6ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.913410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c7fd6-fb4b-402d-88b3-09fb3e4a6ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:54.914156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c7fd6-fb4b-402d-88b3-09fb3e4a6ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:54.915807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c7fd6-fb4b-402d-88b3-09fb3e4a6ac1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:54.928220 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1247/2482] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:54.932505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04027a8c-f321-41bc-b82d-58f1860bad61 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:54.947606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04027a8c-f321-41bc-b82d-58f1860bad61 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:54.955555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a10658-ec84-4ad5-ad6a-b34b545db0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:54.957124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a10658-ec84-4ad5-ad6a-b34b545db0de admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.957489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a10658-ec84-4ad5-ad6a-b34b545db0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.958137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a10658-ec84-4ad5-ad6a-b34b545db0de admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:54.958530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a10658-ec84-4ad5-ad6a-b34b545db0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:54.960309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a10658-ec84-4ad5-ad6a-b34b545db0de admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:54.973637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1236/2483] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:54.979351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c7fd6-fb4b-402d-88b3-09fb3e4a6ac1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:54.991216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c7fd6-fb4b-402d-88b3-09fb3e4a6ac1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:54.995922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3eda0-c502-4078-82d6-ddb0da3fa3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:54.996870 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3eda0-c502-4078-82d6-ddb0da3fa3e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:54.997157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3eda0-c502-4078-82d6-ddb0da3fa3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:54.997435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3eda0-c502-4078-82d6-ddb0da3fa3e7 admin admin] PATH_INFO: `/v3/projects/79c057e14b644edabf55e555d4a18b4b/users/86a14eb1c314449892281892a0f6bd7b/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:55.015842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3eda0-c502-4078-82d6-ddb0da3fa3e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c057e14b644edabf55e555d4a18b4b, user_id=86a14eb1c314449892281892a0f6bd7b, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:55.019131 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3eda0-c502-4078-82d6-ddb0da3fa3e7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:55.033930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1248/2484] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:54 2022] PUT /identity/v3/projects/79c057e14b644edabf55e555d4a18b4b/users/86a14eb1c314449892281892a0f6bd7b/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:55.045094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fa741-8b23-4d0b-ae0e-d3889b19eebd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:55.045457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fa741-8b23-4d0b-ae0e-d3889b19eebd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:55.045743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fa741-8b23-4d0b-ae0e-d3889b19eebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:55.078999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-114fa741-8b23-4d0b-ae0e-d3889b19eebd None None] MFA Rules not processed for user `86a14eb1c314449892281892a0f6bd7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:55.080349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-114fa741-8b23-4d0b-ae0e-d3889b19eebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:55.167569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1237/2485] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:55.179878 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3eda0-c502-4078-82d6-ddb0da3fa3e7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:55.181620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-87a3eda0-c502-4078-82d6-ddb0da3fa3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:55.213762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:55.305619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3eda0-c502-4078-82d6-ddb0da3fa3e7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:55.313850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67e36a-f448-4b31-8559-5fc1cff7b605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:55.315413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67e36a-f448-4b31-8559-5fc1cff7b605 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:55.316206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67e36a-f448-4b31-8559-5fc1cff7b605 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:55.316627 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67e36a-f448-4b31-8559-5fc1cff7b605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:55.327070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67e36a-f448-4b31-8559-5fc1cff7b605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:55.328721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67e36a-f448-4b31-8559-5fc1cff7b605 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:55.338470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1249/2486] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:55.569903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fa741-8b23-4d0b-ae0e-d3889b19eebd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:55.640126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fa741-8b23-4d0b-ae0e-d3889b19eebd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:55.645368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:55.646310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:55.646680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:55.646966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] PATH_INFO: `/v3/users/86a14eb1c314449892281892a0f6bd7b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:55.648231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a14eb1c314449892281892a0f6bd7b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:55.650215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:55.697637 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a14eb1c314449892281892a0f6bd7b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:55.705775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a14eb1c314449892281892a0f6bd7b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:55.713416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a14eb1c314449892281892a0f6bd7b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:55.715304 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:55.732638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a14eb1c314449892281892a0f6bd7b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:55.733377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a14eb1c314449892281892a0f6bd7b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:55.740670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1238/2487] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:55 2022] DELETE /identity/v3/users/86a14eb1c314449892281892a0f6bd7b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:55.744946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d78258-0cb1-4112-b0ae-53614c0f1138 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:55.745460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d78258-0cb1-4112-b0ae-53614c0f1138 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:55.745570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d78258-0cb1-4112-b0ae-53614c0f1138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:55.763696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-40d78258-0cb1-4112-b0ae-53614c0f1138 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:55.765033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-40d78258-0cb1-4112-b0ae-53614c0f1138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:55.824928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1250/2488] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:55.835067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:55.836544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:55.893970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815e6d5-7777-48c3-a3a8-609bf13c44fe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:55.897702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef80d2a-5a66-4e2d-94d5-31b35e613273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:55.898379 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef80d2a-5a66-4e2d-94d5-31b35e613273 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:55.898667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef80d2a-5a66-4e2d-94d5-31b35e613273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:55.898945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef80d2a-5a66-4e2d-94d5-31b35e613273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:55.908436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef80d2a-5a66-4e2d-94d5-31b35e613273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:55.909875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef80d2a-5a66-4e2d-94d5-31b35e613273 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:55.920283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1239/2489] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:56.213310 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d78258-0cb1-4112-b0ae-53614c0f1138 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:56.215248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-40d78258-0cb1-4112-b0ae-53614c0f1138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:56.230836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d78258-0cb1-4112-b0ae-53614c0f1138 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:56.238089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:56.238997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.239621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.240051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] PATH_INFO: `/v3/projects/79c057e14b644edabf55e555d4a18b4b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.241303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c057e14b644edabf55e555d4a18b4b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:56.243385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:56.303430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c057e14b644edabf55e555d4a18b4b', 'request_id': 'req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.310759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c057e14b644edabf55e555d4a18b4b', 'request_id': 'req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.321392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c057e14b644edabf55e555d4a18b4b', 'request_id': 'req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.337970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] The token cache is being invalidate because project 79c057e14b644edabf55e555d4a18b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:56.338273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.338968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.340845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1251/2490] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:56 2022] DELETE /identity/v3/projects/79c057e14b644edabf55e555d4a18b4b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:56.346008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37249ec2-c5e1-4895-b218-63ea8ea9c4db None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.346338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37249ec2-c5e1-4895-b218-63ea8ea9c4db None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.346630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37249ec2-c5e1-4895-b218-63ea8ea9c4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.368206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-37249ec2-c5e1-4895-b218-63ea8ea9c4db None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:56.369528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-37249ec2-c5e1-4895-b218-63ea8ea9c4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:56.415619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:56.417337 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:56.434945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:56.458038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1240/2491] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:56.466030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434f5bf-4e9d-431f-96f0-f5c982d31370 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.466411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434f5bf-4e9d-431f-96f0-f5c982d31370 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.466764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434f5bf-4e9d-431f-96f0-f5c982d31370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.495766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3434f5bf-4e9d-431f-96f0-f5c982d31370 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:56.497339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3434f5bf-4e9d-431f-96f0-f5c982d31370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:56.507204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1241/2492] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:56.512362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434f5bf-4e9d-431f-96f0-f5c982d31370 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:56.524064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434f5bf-4e9d-431f-96f0-f5c982d31370 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:56.533165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c26c90-4dfa-4a90-9384-8eafecc52f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:56.533545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c26c90-4dfa-4a90-9384-8eafecc52f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.533793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511fdf6-fd80-47f6-bfc4-e83f6676dc8b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:56.534143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c26c90-4dfa-4a90-9384-8eafecc52f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.535152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c26c90-4dfa-4a90-9384-8eafecc52f6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.535979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c26c90-4dfa-4a90-9384-8eafecc52f6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:21:56.536953 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c26c90-4dfa-4a90-9384-8eafecc52f6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:56.540958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c26c90-4dfa-4a90-9384-8eafecc52f6e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:56.550926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:56.552048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.553010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.553415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] PATH_INFO: `/v3/users/f31a6a36e475420b82cac82b411e67ba` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.554955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31a6a36e475420b82cac82b411e67ba)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:56.557095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:56.574540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1242/2493] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:21:56.579129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c26c90-4dfa-4a90-9384-8eafecc52f6e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:56.590881 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c26c90-4dfa-4a90-9384-8eafecc52f6e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:56.599466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d92b9-cdd1-4261-80f6-b08e25145b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:56.599466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d92b9-cdd1-4261-80f6-b08e25145b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.599466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d92b9-cdd1-4261-80f6-b08e25145b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.599466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d92b9-cdd1-4261-80f6-b08e25145b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.600473 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d92b9-cdd1-4261-80f6-b08e25145b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:56.602208 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d92b9-cdd1-4261-80f6-b08e25145b9d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:56.626402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31a6a36e475420b82cac82b411e67ba'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.627309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31a6a36e475420b82cac82b411e67ba'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.635837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31a6a36e475420b82cac82b411e67ba'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.638937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1243/2494] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:56 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:21:56.643111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d92b9-cdd1-4261-80f6-b08e25145b9d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:56.644945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-694d92b9-cdd1-4261-80f6-b08e25145b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:56.651184 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31a6a36e475420b82cac82b411e67ba'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.659219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8ec21d9-dffc-46ca-b486-e796875a5691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31a6a36e475420b82cac82b411e67ba'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:56.665621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1252/2495] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:56 2022] DELETE /identity/v3/users/f31a6a36e475420b82cac82b411e67ba => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:21:56.670353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ea315-27be-44c1-96ef-365f38d18a7d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.670727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ea315-27be-44c1-96ef-365f38d18a7d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.671092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ea315-27be-44c1-96ef-365f38d18a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.693148 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c44ea315-27be-44c1-96ef-365f38d18a7d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:56.694369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c44ea315-27be-44c1-96ef-365f38d18a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:56.701642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:56.730712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d92b9-cdd1-4261-80f6-b08e25145b9d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:56.782377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1253/2496] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:56.796449 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ea315-27be-44c1-96ef-365f38d18a7d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:56.799134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c44ea315-27be-44c1-96ef-365f38d18a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:56.807855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cd75f-f606-44b1-9806-e816c42645d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:56.809161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cd75f-f606-44b1-9806-e816c42645d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.809520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cd75f-f606-44b1-9806-e816c42645d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.809868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cd75f-f606-44b1-9806-e816c42645d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.811171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cd75f-f606-44b1-9806-e816c42645d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:56.813252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cd75f-f606-44b1-9806-e816c42645d6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:56.836301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1244/2497] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:56 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:56.841292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cd75f-f606-44b1-9806-e816c42645d6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:56.845409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:56.852553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cd75f-f606-44b1-9806-e816c42645d6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:56.859830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9a32-46ea-4fc0-a6f0-c485783a960b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:56.861562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9a32-46ea-4fc0-a6f0-c485783a960b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.862180 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9a32-46ea-4fc0-a6f0-c485783a960b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.862524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9a32-46ea-4fc0-a6f0-c485783a960b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.864155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb9a32-46ea-4fc0-a6f0-c485783a960b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:56.867859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb9a32-46ea-4fc0-a6f0-c485783a960b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:56.881190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1245/2498] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:56.885103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9a32-46ea-4fc0-a6f0-c485783a960b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:56.885807 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ea315-27be-44c1-96ef-365f38d18a7d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:56.893961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74fdd5-98f6-4f44-ad92-dbeb3c11a5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:56.894651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74fdd5-98f6-4f44-ad92-dbeb3c11a5cb service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.894939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74fdd5-98f6-4f44-ad92-dbeb3c11a5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.895219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74fdd5-98f6-4f44-ad92-dbeb3c11a5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.897016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9a32-46ea-4fc0-a6f0-c485783a960b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:56.902081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc21dd-202f-4ebb-bf38-9a2d92c76b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:56.902751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc21dd-202f-4ebb-bf38-9a2d92c76b4f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.903039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc21dd-202f-4ebb-bf38-9a2d92c76b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.904007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc21dd-202f-4ebb-bf38-9a2d92c76b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.904332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc21dd-202f-4ebb-bf38-9a2d92c76b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:56.905950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc21dd-202f-4ebb-bf38-9a2d92c76b4f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:56.911690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74fdd5-98f6-4f44-ad92-dbeb3c11a5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:56.913333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74fdd5-98f6-4f44-ad92-dbeb3c11a5cb service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:56.923300 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1246/2499] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:56.925000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc21dd-202f-4ebb-bf38-9a2d92c76b4f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:56.926923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1254/2500] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:56.939139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc21dd-202f-4ebb-bf38-9a2d92c76b4f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:56.944890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36c6e8-f6e5-41f2-acbc-974c15c1d4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:56.946250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36c6e8-f6e5-41f2-acbc-974c15c1d4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:56.946718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36c6e8-f6e5-41f2-acbc-974c15c1d4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:56.947001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36c6e8-f6e5-41f2-acbc-974c15c1d4fe admin admin] PATH_INFO: `/v3/projects/c1683838ed2b40729d6fbf2440f5f3ce/users/0658900e732b4c1e8ca37d7e4a85eca8/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:56.965249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36c6e8-f6e5-41f2-acbc-974c15c1d4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1683838ed2b40729d6fbf2440f5f3ce, user_id=0658900e732b4c1e8ca37d7e4a85eca8, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:56.970088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36c6e8-f6e5-41f2-acbc-974c15c1d4fe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:56.987751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1247/2501] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:56 2022] PUT /identity/v3/projects/c1683838ed2b40729d6fbf2440f5f3ce/users/0658900e732b4c1e8ca37d7e4a85eca8/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:21:57.002461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d936b-729a-4a94-9172-52409f4d5e30 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.002842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d936b-729a-4a94-9172-52409f4d5e30 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.002988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d936b-729a-4a94-9172-52409f4d5e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.036693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-780d936b-729a-4a94-9172-52409f4d5e30 None None] MFA Rules not processed for user `0658900e732b4c1e8ca37d7e4a85eca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:57.037814 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-780d936b-729a-4a94-9172-52409f4d5e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:57.144862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1255/2502] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:57.149296 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36c6e8-f6e5-41f2-acbc-974c15c1d4fe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:57.229469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36c6e8-f6e5-41f2-acbc-974c15c1d4fe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:57.235844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854cfc6-b1e5-4080-86f0-71af1b7fe843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:57.236662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854cfc6-b1e5-4080-86f0-71af1b7fe843 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.236955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854cfc6-b1e5-4080-86f0-71af1b7fe843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.237230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854cfc6-b1e5-4080-86f0-71af1b7fe843 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.238169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854cfc6-b1e5-4080-86f0-71af1b7fe843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:57.240190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854cfc6-b1e5-4080-86f0-71af1b7fe843 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:57.270125 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d936b-729a-4a94-9172-52409f4d5e30 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:57.271773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-780d936b-729a-4a94-9172-52409f4d5e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:57.287927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1248/2503] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:57 2022] POST /identity/v3/projects => generated 369 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:21:57.291765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854cfc6-b1e5-4080-86f0-71af1b7fe843 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:57.297252 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d936b-729a-4a94-9172-52409f4d5e30 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:57.331354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:57.379920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:57.381031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.381417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.381518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854cfc6-b1e5-4080-86f0-71af1b7fe843 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:57.381867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] PATH_INFO: `/v3/projects/e5f2d9d01a204b5da03a2c7e1eee80e7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.383173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f2d9d01a204b5da03a2c7e1eee80e7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:57.385321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:57.389923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c49ddc-13ff-4d76-bf3e-9cbfc24ca974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:57.390617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c49ddc-13ff-4d76-bf3e-9cbfc24ca974 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.390909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c49ddc-13ff-4d76-bf3e-9cbfc24ca974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.391192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c49ddc-13ff-4d76-bf3e-9cbfc24ca974 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.392586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c49ddc-13ff-4d76-bf3e-9cbfc24ca974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:57.395600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c49ddc-13ff-4d76-bf3e-9cbfc24ca974 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:57.422955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1249/2504] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:57 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:57.428827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c49ddc-13ff-4d76-bf3e-9cbfc24ca974 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:57.440513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c49ddc-13ff-4d76-bf3e-9cbfc24ca974 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:57.450246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f2d9d01a204b5da03a2c7e1eee80e7', 'request_id': 'req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.458316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f2d9d01a204b5da03a2c7e1eee80e7', 'request_id': 'req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.461180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:57.467919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f2d9d01a204b5da03a2c7e1eee80e7', 'request_id': 'req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.492122 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] The token cache is being invalidate because project e5f2d9d01a204b5da03a2c7e1eee80e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:57.492507 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.493716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.495678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1256/2505] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:57 2022] DELETE /identity/v3/projects/e5f2d9d01a204b5da03a2c7e1eee80e7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:21:57.500385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:57.502318 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:57.520842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82613dcc-9fa3-48d0-9aaa-45e0a7733b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:57.522307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82613dcc-9fa3-48d0-9aaa-45e0a7733b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.522307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82613dcc-9fa3-48d0-9aaa-45e0a7733b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.522307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82613dcc-9fa3-48d0-9aaa-45e0a7733b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.523501 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82613dcc-9fa3-48d0-9aaa-45e0a7733b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:57.524734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82613dcc-9fa3-48d0-9aaa-45e0a7733b38 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:57.536711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1250/2506] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:57.547086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82613dcc-9fa3-48d0-9aaa-45e0a7733b38 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:57.549793 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-82613dcc-9fa3-48d0-9aaa-45e0a7733b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:57.562475 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82613dcc-9fa3-48d0-9aaa-45e0a7733b38 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:57.567287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a13b57-d76f-4881-b8f7-e68bb20df9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:57.568191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a13b57-d76f-4881-b8f7-e68bb20df9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.568477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a13b57-d76f-4881-b8f7-e68bb20df9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.568755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a13b57-d76f-4881-b8f7-e68bb20df9ae admin admin] PATH_INFO: `/v3/projects/2a4ca21f1c424ae982b8927ea8e89d00/users/aa9b3246ffb449c3ab5fd282a70fd0ae/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.587970 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a13b57-d76f-4881-b8f7-e68bb20df9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4ca21f1c424ae982b8927ea8e89d00, user_id=aa9b3246ffb449c3ab5fd282a70fd0ae, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:57.590866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a13b57-d76f-4881-b8f7-e68bb20df9ae admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:57.606930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9ac5f-9e09-420b-b195-4d2bfb4a72de admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:57.612328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1251/2507] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:57 2022] PUT /identity/v3/projects/2a4ca21f1c424ae982b8927ea8e89d00/users/aa9b3246ffb449c3ab5fd282a70fd0ae/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:57.618830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a13b57-d76f-4881-b8f7-e68bb20df9ae admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:57.685282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:57.686211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.686489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.686766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] PATH_INFO: `/v3/users/c2ee3d2286224fd9aa1d430ad5a9bbe3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.688135 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ee3d2286224fd9aa1d430ad5a9bbe3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:57.690628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:57.702472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a13b57-d76f-4881-b8f7-e68bb20df9ae admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:57.709120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31305b5a-6f06-4700-af97-fa7bd08ce520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:57.709904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31305b5a-6f06-4700-af97-fa7bd08ce520 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.710265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31305b5a-6f06-4700-af97-fa7bd08ce520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.710614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31305b5a-6f06-4700-af97-fa7bd08ce520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.711480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31305b5a-6f06-4700-af97-fa7bd08ce520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:57.713284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31305b5a-6f06-4700-af97-fa7bd08ce520 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:57.725332 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1252/2508] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:21:57.729747 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31305b5a-6f06-4700-af97-fa7bd08ce520 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:57.743009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31305b5a-6f06-4700-af97-fa7bd08ce520 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:57.749121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b9fdd-2614-411b-8112-6337e5d511b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:57.750023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b9fdd-2614-411b-8112-6337e5d511b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.750306 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b9fdd-2614-411b-8112-6337e5d511b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.750582 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b9fdd-2614-411b-8112-6337e5d511b6 admin admin] PATH_INFO: `/v3/projects/2a4ca21f1c424ae982b8927ea8e89d00/users/aa9b3246ffb449c3ab5fd282a70fd0ae/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.753168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b9fdd-2614-411b-8112-6337e5d511b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4ca21f1c424ae982b8927ea8e89d00, user_id=aa9b3246ffb449c3ab5fd282a70fd0ae, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:57.755877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b9fdd-2614-411b-8112-6337e5d511b6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:57.761571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ee3d2286224fd9aa1d430ad5a9bbe3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.762544 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ee3d2286224fd9aa1d430ad5a9bbe3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.771548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1253/2509] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:57 2022] PUT /identity/v3/projects/2a4ca21f1c424ae982b8927ea8e89d00/users/aa9b3246ffb449c3ab5fd282a70fd0ae/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:57.773296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ee3d2286224fd9aa1d430ad5a9bbe3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.782242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925c408-45bf-46a5-8050-9d272333ad42 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.782826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925c408-45bf-46a5-8050-9d272333ad42 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.783302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925c408-45bf-46a5-8050-9d272333ad42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:57.789298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ee3d2286224fd9aa1d430ad5a9bbe3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.792684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:57.805664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ee3d2286224fd9aa1d430ad5a9bbe3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:57.808617 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1257/2510] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:57 2022] DELETE /identity/v3/users/c2ee3d2286224fd9aa1d430ad5a9bbe3 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:21:57.820693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d925c408-45bf-46a5-8050-9d272333ad42 None None] MFA Rules not processed for user `aa9b3246ffb449c3ab5fd282a70fd0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:57.821908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d925c408-45bf-46a5-8050-9d272333ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:57.901936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1254/2511] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:57 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:21:57.913145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:57.914844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:57.935572 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:57.982517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b41205-adb8-4af3-bb58-bac031f67ff6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:57.987953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195affe3-bf7f-4432-895c-80ba9ae57730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:57.988739 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195affe3-bf7f-4432-895c-80ba9ae57730 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:57.989126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195affe3-bf7f-4432-895c-80ba9ae57730 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:57.989753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195affe3-bf7f-4432-895c-80ba9ae57730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:58.000256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195affe3-bf7f-4432-895c-80ba9ae57730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:58.004308 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195affe3-bf7f-4432-895c-80ba9ae57730 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:58.015728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1258/2512] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:21:57 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:21:58.213206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925c408-45bf-46a5-8050-9d272333ad42 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:58.221555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d925c408-45bf-46a5-8050-9d272333ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:58.303518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925c408-45bf-46a5-8050-9d272333ad42 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:58.308200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:58.309160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:58.309701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:58.309701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] PATH_INFO: `/v3/projects/14a902f69a974cb9bc4c4bf95830fde5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:58.311736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a902f69a974cb9bc4c4bf95830fde5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:58.314700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:58.371207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a902f69a974cb9bc4c4bf95830fde5', 'request_id': 'req-b6974117-4144-4cab-8278-472f2216b6fc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.384112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a902f69a974cb9bc4c4bf95830fde5', 'request_id': 'req-b6974117-4144-4cab-8278-472f2216b6fc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.390576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a902f69a974cb9bc4c4bf95830fde5', 'request_id': 'req-b6974117-4144-4cab-8278-472f2216b6fc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.405577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] The token cache is being invalidate because project 14a902f69a974cb9bc4c4bf95830fde5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:58.407563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.407939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.412571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1255/2513] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:58 2022] DELETE /identity/v3/projects/14a902f69a974cb9bc4c4bf95830fde5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:58.414159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195affe3-bf7f-4432-895c-80ba9ae57730 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:58.427843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195affe3-bf7f-4432-895c-80ba9ae57730 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:58.580574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:58.583928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:58.584245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:58.584607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] PATH_INFO: `/v3/users/83ce9ee164b64753936ffd1ab20c5731` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:58.586007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ce9ee164b64753936ffd1ab20c5731)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:58.588668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:58.668291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ce9ee164b64753936ffd1ab20c5731'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.683826 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ce9ee164b64753936ffd1ab20c5731'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.697833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ce9ee164b64753936ffd1ab20c5731'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.724397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ce9ee164b64753936ffd1ab20c5731'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.738876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ee1ea58-063e-4e4e-9c2b-5629d981bc73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ce9ee164b64753936ffd1ab20c5731'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:58.749264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1259/2514] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:58 2022] DELETE /identity/v3/users/83ce9ee164b64753936ffd1ab20c5731 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:59.211715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:59.213959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:59.229971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6974117-4144-4cab-8278-472f2216b6fc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:59.262097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:59.326996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:59.327763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.328063 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.328330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] PATH_INFO: `/v3/projects/ad6866112de94a239115ee9b48304b46` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.330285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6866112de94a239115ee9b48304b46)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:59.332200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:59.403113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6866112de94a239115ee9b48304b46', 'request_id': 'req-03b96c51-b87d-4539-af7a-bc07404ad527'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.415013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6866112de94a239115ee9b48304b46', 'request_id': 'req-03b96c51-b87d-4539-af7a-bc07404ad527'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.427227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6866112de94a239115ee9b48304b46', 'request_id': 'req-03b96c51-b87d-4539-af7a-bc07404ad527'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.448367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] The token cache is being invalidate because project ad6866112de94a239115ee9b48304b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:21:59.448675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.449370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-03b96c51-b87d-4539-af7a-bc07404ad527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.451207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1256/2515] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:21:59 2022] DELETE /identity/v3/projects/ad6866112de94a239115ee9b48304b46 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:59.462269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13103c15-bce1-466b-94d5-628941f4f014 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.462756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13103c15-bce1-466b-94d5-628941f4f014 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.462858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13103c15-bce1-466b-94d5-628941f4f014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.487374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-13103c15-bce1-466b-94d5-628941f4f014 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:59.488738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-13103c15-bce1-466b-94d5-628941f4f014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:59.525962 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:59.562666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1260/2516] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:59.571757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21555e1-20e7-4dbe-a20e-fae69ba6b6e1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.572056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21555e1-20e7-4dbe-a20e-fae69ba6b6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.572351 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21555e1-20e7-4dbe-a20e-fae69ba6b6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.594683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c21555e1-20e7-4dbe-a20e-fae69ba6b6e1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:59.594683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c21555e1-20e7-4dbe-a20e-fae69ba6b6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:59.602206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1257/2517] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:21:59.606761 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13103c15-bce1-466b-94d5-628941f4f014 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:59.619842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13103c15-bce1-466b-94d5-628941f4f014 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:59.627068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5343d-937c-4ce9-8a49-c892ac8beafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:59.627789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5343d-937c-4ce9-8a49-c892ac8beafe admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.628075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5343d-937c-4ce9-8a49-c892ac8beafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.628352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5343d-937c-4ce9-8a49-c892ac8beafe admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.629091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5343d-937c-4ce9-8a49-c892ac8beafe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:21:59.629401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5343d-937c-4ce9-8a49-c892ac8beafe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:59.631044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5343d-937c-4ce9-8a49-c892ac8beafe admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:59.645712 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1261/2518] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:21:59.650093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21555e1-20e7-4dbe-a20e-fae69ba6b6e1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:59.659600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21555e1-20e7-4dbe-a20e-fae69ba6b6e1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:59.664603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f48b7f-0350-45f9-9bff-80f370b0fdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:59.665271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f48b7f-0350-45f9-9bff-80f370b0fdd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.665555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f48b7f-0350-45f9-9bff-80f370b0fdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.665829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f48b7f-0350-45f9-9bff-80f370b0fdd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.666532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f48b7f-0350-45f9-9bff-80f370b0fdd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:59.668531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f48b7f-0350-45f9-9bff-80f370b0fdd5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:59.698550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1258/2519] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:21:59 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:59.702904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5343d-937c-4ce9-8a49-c892ac8beafe admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:59.713260 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5343d-937c-4ce9-8a49-c892ac8beafe admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:59.783572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f48b7f-0350-45f9-9bff-80f370b0fdd5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:59.786094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66f48b7f-0350-45f9-9bff-80f370b0fdd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:59.788011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8e254-8c25-4450-9e58-111d52c4f080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:59.789045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8e254-8c25-4450-9e58-111d52c4f080 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.789520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8e254-8c25-4450-9e58-111d52c4f080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.789987 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8e254-8c25-4450-9e58-111d52c4f080 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.791157 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8e254-8c25-4450-9e58-111d52c4f080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:59.794629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8e254-8c25-4450-9e58-111d52c4f080 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:59.806768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f48b7f-0350-45f9-9bff-80f370b0fdd5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:59.814419 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:59.816157 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.816695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.817240 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] PATH_INFO: `/v3/users/0658900e732b4c1e8ca37d7e4a85eca8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.819128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0658900e732b4c1e8ca37d7e4a85eca8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:59.822531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:59.826841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1262/2520] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:21:59 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:21:59.831026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8e254-8c25-4450-9e58-111d52c4f080 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:59.841091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8e254-8c25-4450-9e58-111d52c4f080 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:59.847580 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88e45f-61c9-4a7b-8b9f-c789d7114f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:59.848179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88e45f-61c9-4a7b-8b9f-c789d7114f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.848466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88e45f-61c9-4a7b-8b9f-c789d7114f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.848743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88e45f-61c9-4a7b-8b9f-c789d7114f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.849496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88e45f-61c9-4a7b-8b9f-c789d7114f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:59.851093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88e45f-61c9-4a7b-8b9f-c789d7114f57 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:59.864502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1263/2521] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:59.869149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88e45f-61c9-4a7b-8b9f-c789d7114f57 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:59.881012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88e45f-61c9-4a7b-8b9f-c789d7114f57 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:59.884090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0658900e732b4c1e8ca37d7e4a85eca8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.885709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e596c9-f052-4ae6-8be5-6c334545c398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:59.886374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e596c9-f052-4ae6-8be5-6c334545c398 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.886666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e596c9-f052-4ae6-8be5-6c334545c398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.886944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e596c9-f052-4ae6-8be5-6c334545c398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.887744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e596c9-f052-4ae6-8be5-6c334545c398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:59.889805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e596c9-f052-4ae6-8be5-6c334545c398 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:59.893482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0658900e732b4c1e8ca37d7e4a85eca8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.902122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0658900e732b4c1e8ca37d7e4a85eca8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.902414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1264/2522] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:21:59.906536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e596c9-f052-4ae6-8be5-6c334545c398 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:21:59.916230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:21:59.916774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e596c9-f052-4ae6-8be5-6c334545c398 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:21:59.918333 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0658900e732b4c1e8ca37d7e4a85eca8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.919283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2f5f9c77-1c00-44bf-98ee-f9d4c33706d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0658900e732b4c1e8ca37d7e4a85eca8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:21:59.923940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e95e9-5f5f-4d85-9235-b700900c04a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:21:59.925773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e95e9-5f5f-4d85-9235-b700900c04a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.926240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e95e9-5f5f-4d85-9235-b700900c04a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.926652 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e95e9-5f5f-4d85-9235-b700900c04a4 admin admin] PATH_INFO: `/v3/projects/c6b4e932b06543e5ae9f8e125a3f5b53/users/21381d26be9847de9dc6c71bff9fadc2/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.929334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1259/2523] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:21:59 2022] DELETE /identity/v3/users/0658900e732b4c1e8ca37d7e4a85eca8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:21:59.934431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c68ed2-8df3-411c-ab0a-690f32691ee6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.934952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c68ed2-8df3-411c-ab0a-690f32691ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.935328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c68ed2-8df3-411c-ab0a-690f32691ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:21:59.947705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e95e9-5f5f-4d85-9235-b700900c04a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b4e932b06543e5ae9f8e125a3f5b53, user_id=21381d26be9847de9dc6c71bff9fadc2, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:21:59.950380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e95e9-5f5f-4d85-9235-b700900c04a4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:21:59.955762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-10c68ed2-8df3-411c-ab0a-690f32691ee6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:21:59.957395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-10c68ed2-8df3-411c-ab0a-690f32691ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:21:59.964981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1265/2524] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:21:59 2022] PUT /identity/v3/projects/c6b4e932b06543e5ae9f8e125a3f5b53/users/21381d26be9847de9dc6c71bff9fadc2/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:21:59.977352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1744603-48be-45e1-9f99-0b8bf30413b2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:21:59.977950 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1744603-48be-45e1-9f99-0b8bf30413b2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:21:59.978522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1744603-48be-45e1-9f99-0b8bf30413b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.009409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d1744603-48be-45e1-9f99-0b8bf30413b2 None None] MFA Rules not processed for user `21381d26be9847de9dc6c71bff9fadc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:00.010652 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d1744603-48be-45e1-9f99-0b8bf30413b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:00.042485 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1260/2525] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:00.058545 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c68ed2-8df3-411c-ab0a-690f32691ee6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.061365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-10c68ed2-8df3-411c-ab0a-690f32691ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:00.092133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:00.100458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1266/2526] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:21:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:00.105048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1744603-48be-45e1-9f99-0b8bf30413b2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.108483 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d1744603-48be-45e1-9f99-0b8bf30413b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:00.132441 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1744603-48be-45e1-9f99-0b8bf30413b2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.135450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c68ed2-8df3-411c-ab0a-690f32691ee6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.139221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60cb54-28fd-41a8-af14-c8d3f2700266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:00.139921 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60cb54-28fd-41a8-af14-c8d3f2700266 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.140204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60cb54-28fd-41a8-af14-c8d3f2700266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.140321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5de28-3eec-4ebf-8f06-a78987468a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:00.140502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60cb54-28fd-41a8-af14-c8d3f2700266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.141423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5de28-3eec-4ebf-8f06-a78987468a7c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.141933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5de28-3eec-4ebf-8f06-a78987468a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.142503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5de28-3eec-4ebf-8f06-a78987468a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.143759 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5de28-3eec-4ebf-8f06-a78987468a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:00.147584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5de28-3eec-4ebf-8f06-a78987468a7c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:00.152620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60cb54-28fd-41a8-af14-c8d3f2700266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:00.153718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60cb54-28fd-41a8-af14-c8d3f2700266 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:00.165765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1261/2527] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:00.191445 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1267/2528] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:00 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:00.194858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60cb54-28fd-41a8-af14-c8d3f2700266 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.205776 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60cb54-28fd-41a8-af14-c8d3f2700266 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.278748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb85c81-af93-48a2-8e98-93d2fc06049a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:00.279510 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb85c81-af93-48a2-8e98-93d2fc06049a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.279754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb85c81-af93-48a2-8e98-93d2fc06049a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.280037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb85c81-af93-48a2-8e98-93d2fc06049a admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.280742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb85c81-af93-48a2-8e98-93d2fc06049a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:00.283460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb85c81-af93-48a2-8e98-93d2fc06049a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:00.310160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1262/2529] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:00 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:00.314788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5de28-3eec-4ebf-8f06-a78987468a7c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.330334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5de28-3eec-4ebf-8f06-a78987468a7c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.337728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ab031-b1e8-49db-88d6-8a0600a3d4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:00.338856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ab031-b1e8-49db-88d6-8a0600a3d4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.339391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ab031-b1e8-49db-88d6-8a0600a3d4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.339911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ab031-b1e8-49db-88d6-8a0600a3d4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.340865 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ab031-b1e8-49db-88d6-8a0600a3d4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:00.342790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ab031-b1e8-49db-88d6-8a0600a3d4d2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:00.358531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1268/2530] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:00.360006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb85c81-af93-48a2-8e98-93d2fc06049a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.371868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb85c81-af93-48a2-8e98-93d2fc06049a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.377014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0256c-1602-47cf-82fa-664b3fbcc138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:00.378220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0256c-1602-47cf-82fa-664b3fbcc138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.378420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0256c-1602-47cf-82fa-664b3fbcc138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.378740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0256c-1602-47cf-82fa-664b3fbcc138 admin admin] PATH_INFO: `/v3/projects/157674d652dd4236b571b2b0f78fd956/users/d15e8e272a474924acc845f8088449a2/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.400288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0256c-1602-47cf-82fa-664b3fbcc138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157674d652dd4236b571b2b0f78fd956, user_id=d15e8e272a474924acc845f8088449a2, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:00.403108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0256c-1602-47cf-82fa-664b3fbcc138 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:00.419108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1263/2531] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:00 2022] PUT /identity/v3/projects/157674d652dd4236b571b2b0f78fd956/users/d15e8e272a474924acc845f8088449a2/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:00.423118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ab031-b1e8-49db-88d6-8a0600a3d4d2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.433313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ab031-b1e8-49db-88d6-8a0600a3d4d2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.522199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff548f20-88ab-4948-a381-438219fbfe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:00.523168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff548f20-88ab-4948-a381-438219fbfe20 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.525122 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff548f20-88ab-4948-a381-438219fbfe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.525631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff548f20-88ab-4948-a381-438219fbfe20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.526946 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff548f20-88ab-4948-a381-438219fbfe20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:00.529001 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff548f20-88ab-4948-a381-438219fbfe20 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:00.536955 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0256c-1602-47cf-82fa-664b3fbcc138 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.538920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cfb0256c-1602-47cf-82fa-664b3fbcc138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:00.550287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:00.556406 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1269/2532] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:00.561355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f8484-8ea9-402c-8fab-572fdd00af2d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.561930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f8484-8ea9-402c-8fab-572fdd00af2d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.561930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f8484-8ea9-402c-8fab-572fdd00af2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.574686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0256c-1602-47cf-82fa-664b3fbcc138 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.580895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:00.581782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.582070 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.582408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] PATH_INFO: `/v3/projects/c1683838ed2b40729d6fbf2440f5f3ce` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.583701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1683838ed2b40729d6fbf2440f5f3ce)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:00.586951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:00.617177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5b8f8484-8ea9-402c-8fab-572fdd00af2d None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:00.618437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5b8f8484-8ea9-402c-8fab-572fdd00af2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:00.650446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1683838ed2b40729d6fbf2440f5f3ce', 'request_id': 'req-f5a8d233-a8a3-48d4-9d32-389814278c41'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.660027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1683838ed2b40729d6fbf2440f5f3ce', 'request_id': 'req-f5a8d233-a8a3-48d4-9d32-389814278c41'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.669038 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1683838ed2b40729d6fbf2440f5f3ce', 'request_id': 'req-f5a8d233-a8a3-48d4-9d32-389814278c41'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.686788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1270/2533] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 22 11:22:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:00.687328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] The token cache is being invalidate because project c1683838ed2b40729d6fbf2440f5f3ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:00.687640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.687993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f8484-8ea9-402c-8fab-572fdd00af2d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.689027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.694316 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1264/2534] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:00 2022] DELETE /identity/v3/projects/c1683838ed2b40729d6fbf2440f5f3ce => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:00.695719 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.698226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:00.701928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f8484-8ea9-402c-8fab-572fdd00af2d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.782737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:00.790530 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8d233-a8a3-48d4-9d32-389814278c41 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.792569 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209cb74-9bfd-4c32-9573-9e7dbdd35c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:00.793702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209cb74-9bfd-4c32-9573-9e7dbdd35c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.794202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209cb74-9bfd-4c32-9573-9e7dbdd35c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.794665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209cb74-9bfd-4c32-9573-9e7dbdd35c80 admin admin] PATH_INFO: `/v3/projects/157674d652dd4236b571b2b0f78fd956/users/d15e8e272a474924acc845f8088449a2/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.797132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209cb74-9bfd-4c32-9573-9e7dbdd35c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157674d652dd4236b571b2b0f78fd956, user_id=d15e8e272a474924acc845f8088449a2, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:00.797301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:00.798488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.798783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.799068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] PATH_INFO: `/v3/users/aa9b3246ffb449c3ab5fd282a70fd0ae` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.800334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9b3246ffb449c3ab5fd282a70fd0ae)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:00.800490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209cb74-9bfd-4c32-9573-9e7dbdd35c80 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:00.802315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:00.812873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1271/2535] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:00 2022] PUT /identity/v3/projects/157674d652dd4236b571b2b0f78fd956/users/d15e8e272a474924acc845f8088449a2/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:00.814575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209cb74-9bfd-4c32-9573-9e7dbdd35c80 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:00.816353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5209cb74-9bfd-4c32-9573-9e7dbdd35c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:00.876773 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9b3246ffb449c3ab5fd282a70fd0ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.888167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9b3246ffb449c3ab5fd282a70fd0ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.896859 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9b3246ffb449c3ab5fd282a70fd0ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.910197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9b3246ffb449c3ab5fd282a70fd0ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.910936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cebcd87c-2be1-419e-a532-cc8eb00bfe79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9b3246ffb449c3ab5fd282a70fd0ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:00.917938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209cb74-9bfd-4c32-9573-9e7dbdd35c80 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:00.918725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1265/2536] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:00 2022] DELETE /identity/v3/users/aa9b3246ffb449c3ab5fd282a70fd0ae => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:00.920123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de48ed8-4e18-4e0f-a160-72b131b37694 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:00.920410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de48ed8-4e18-4e0f-a160-72b131b37694 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:00.920697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de48ed8-4e18-4e0f-a160-72b131b37694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:00.927519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:00.957675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0de48ed8-4e18-4e0f-a160-72b131b37694 None None] MFA Rules not processed for user `d15e8e272a474924acc845f8088449a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:00.959018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0de48ed8-4e18-4e0f-a160-72b131b37694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:01.024297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:01.025249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:01.025682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:01.026103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:01.028961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:01.057567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1266/2537] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:01.073403 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de48ed8-4e18-4e0f-a160-72b131b37694 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:01.075080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0de48ed8-4e18-4e0f-a160-72b131b37694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:01.122434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:01.124275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:01.137591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1272/2538] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:22:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:22:01.176567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de48ed8-4e18-4e0f-a160-72b131b37694 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:01.182910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e5df9-60d3-4bc5-954f-601eb76f72d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:01.183671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e5df9-60d3-4bc5-954f-601eb76f72d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:01.187330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e5df9-60d3-4bc5-954f-601eb76f72d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:01.187775 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e5df9-60d3-4bc5-954f-601eb76f72d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:01.199105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e5df9-60d3-4bc5-954f-601eb76f72d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:01.201178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e5df9-60d3-4bc5-954f-601eb76f72d3 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:01.221880 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1267/2539] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:01.283877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:01.286492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:01.421620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:01.486791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d0524-633d-48b4-b593-30b83c9ea0bc service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:01.493881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:01.495727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:01.497400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:01.497999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] PATH_INFO: `/v3/projects/2a4ca21f1c424ae982b8927ea8e89d00` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:01.502707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4ca21f1c424ae982b8927ea8e89d00)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:01.506663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:01.585239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4ca21f1c424ae982b8927ea8e89d00', 'request_id': 'req-4eebace4-a5e9-4570-865a-ea0cd1c66fce'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:01.612477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4ca21f1c424ae982b8927ea8e89d00', 'request_id': 'req-4eebace4-a5e9-4570-865a-ea0cd1c66fce'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:01.625337 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4ca21f1c424ae982b8927ea8e89d00', 'request_id': 'req-4eebace4-a5e9-4570-865a-ea0cd1c66fce'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:01.649254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] The token cache is being invalidate because project 2a4ca21f1c424ae982b8927ea8e89d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:01.649555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:01.650215 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:01.652784 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1273/2540] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:01 2022] DELETE /identity/v3/projects/2a4ca21f1c424ae982b8927ea8e89d00 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:01.658055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a0e67-ffec-4e30-9852-c61b93edf1bf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:01.658544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a0e67-ffec-4e30-9852-c61b93edf1bf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:01.658681 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a0e67-ffec-4e30-9852-c61b93edf1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:01.681094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-841a0e67-ffec-4e30-9852-c61b93edf1bf None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:01.682336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-841a0e67-ffec-4e30-9852-c61b93edf1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:01.742202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:01.749972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:01.772113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1268/2541] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:01.785334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783b8a0-6099-488e-9ce6-bce2d56b86a7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:01.785981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783b8a0-6099-488e-9ce6-bce2d56b86a7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:01.786430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783b8a0-6099-488e-9ce6-bce2d56b86a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:01.817039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1783b8a0-6099-488e-9ce6-bce2d56b86a7 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:01.818800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1783b8a0-6099-488e-9ce6-bce2d56b86a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:01.831424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1269/2542] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:01.838050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783b8a0-6099-488e-9ce6-bce2d56b86a7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:01.856643 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783b8a0-6099-488e-9ce6-bce2d56b86a7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:01.863126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81579e38-7b1d-4a48-82d4-71513efdf7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:01.863851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81579e38-7b1d-4a48-82d4-71513efdf7be admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:01.864581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81579e38-7b1d-4a48-82d4-71513efdf7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:01.864868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81579e38-7b1d-4a48-82d4-71513efdf7be admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:01.865896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81579e38-7b1d-4a48-82d4-71513efdf7be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:22:01.866665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81579e38-7b1d-4a48-82d4-71513efdf7be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:01.869436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81579e38-7b1d-4a48-82d4-71513efdf7be admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:01.888417 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1270/2543] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:22:01.895487 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81579e38-7b1d-4a48-82d4-71513efdf7be admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:01.912913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81579e38-7b1d-4a48-82d4-71513efdf7be admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:01.919844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f0896-29de-49ad-8572-b75db9d31ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:01.920872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f0896-29de-49ad-8572-b75db9d31ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:01.921285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f0896-29de-49ad-8572-b75db9d31ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:01.921875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f0896-29de-49ad-8572-b75db9d31ae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:01.923122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f0896-29de-49ad-8572-b75db9d31ae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:01.925579 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f0896-29de-49ad-8572-b75db9d31ae8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:01.935308 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebace4-a5e9-4570-865a-ea0cd1c66fce admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:01.942358 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:01.943442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:01.943732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:01.944011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] PATH_INFO: `/v3/users/21381d26be9847de9dc6c71bff9fadc2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:01.945791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21381d26be9847de9dc6c71bff9fadc2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:01.947909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:01.957134 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:02.007625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1271/2544] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:01 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:02.010976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f0896-29de-49ad-8572-b75db9d31ae8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.028540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21381d26be9847de9dc6c71bff9fadc2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.029675 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21381d26be9847de9dc6c71bff9fadc2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.041889 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21381d26be9847de9dc6c71bff9fadc2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.056567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:02.059699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21381d26be9847de9dc6c71bff9fadc2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.073598 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15fb090d-85c2-4267-b3ff-5aa3faeebfe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21381d26be9847de9dc6c71bff9fadc2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.080982 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1274/2545] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:01 2022] DELETE /identity/v3/users/21381d26be9847de9dc6c71bff9fadc2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:02.086076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101fd52-2267-4631-956e-858eb9de9632 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.086619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101fd52-2267-4631-956e-858eb9de9632 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.086906 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101fd52-2267-4631-956e-858eb9de9632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.095424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f0896-29de-49ad-8572-b75db9d31ae8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.127491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e101fd52-2267-4631-956e-858eb9de9632 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:02.130036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e101fd52-2267-4631-956e-858eb9de9632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:02.173974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a93ff-8bd0-4c1a-8786-d49700e2b662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.175245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a93ff-8bd0-4c1a-8786-d49700e2b662 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.175654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a93ff-8bd0-4c1a-8786-d49700e2b662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.175925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a93ff-8bd0-4c1a-8786-d49700e2b662 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.176651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a93ff-8bd0-4c1a-8786-d49700e2b662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.179192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a93ff-8bd0-4c1a-8786-d49700e2b662 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.201122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1272/2546] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:02 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:02.207780 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a93ff-8bd0-4c1a-8786-d49700e2b662 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.210394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8a4a93ff-8bd0-4c1a-8786-d49700e2b662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:02.218998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1275/2547] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:02.230155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a93ff-8bd0-4c1a-8786-d49700e2b662 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.232174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101fd52-2267-4631-956e-858eb9de9632 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.233910 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e101fd52-2267-4631-956e-858eb9de9632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:02.237940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e82ca-583e-4da3-a83b-66a8b345ad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.239014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e82ca-583e-4da3-a83b-66a8b345ad13 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.240058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e82ca-583e-4da3-a83b-66a8b345ad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.240524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e82ca-583e-4da3-a83b-66a8b345ad13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.241835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e82ca-583e-4da3-a83b-66a8b345ad13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.244567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e82ca-583e-4da3-a83b-66a8b345ad13 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.264317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1273/2548] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:02.270834 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e82ca-583e-4da3-a83b-66a8b345ad13 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.281221 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e82ca-583e-4da3-a83b-66a8b345ad13 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.286409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363d586-282b-4805-b9a0-4d5ac936d96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.287193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363d586-282b-4805-b9a0-4d5ac936d96f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.287517 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363d586-282b-4805-b9a0-4d5ac936d96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.287807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363d586-282b-4805-b9a0-4d5ac936d96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.288543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363d586-282b-4805-b9a0-4d5ac936d96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.290187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363d586-282b-4805-b9a0-4d5ac936d96f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.296837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101fd52-2267-4631-956e-858eb9de9632 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.300900 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37260d0-2f8b-4377-b7f7-b8761fabebc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.301568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37260d0-2f8b-4377-b7f7-b8761fabebc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.301862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37260d0-2f8b-4377-b7f7-b8761fabebc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.302144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37260d0-2f8b-4377-b7f7-b8761fabebc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.307159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1274/2549] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:02.312710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363d586-282b-4805-b9a0-4d5ac936d96f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.312888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37260d0-2f8b-4377-b7f7-b8761fabebc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.315566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37260d0-2f8b-4377-b7f7-b8761fabebc3 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.326218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363d586-282b-4805-b9a0-4d5ac936d96f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.332388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a173fd-57a1-4bdd-90da-e9547537d608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.333438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a173fd-57a1-4bdd-90da-e9547537d608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.333750 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a173fd-57a1-4bdd-90da-e9547537d608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.334034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a173fd-57a1-4bdd-90da-e9547537d608 admin admin] PATH_INFO: `/v3/projects/1bff62bfcb0946eeac42ebd190731351/users/9212efdd0bc840248786839746836571/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.337432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1276/2550] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:02.351964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a173fd-57a1-4bdd-90da-e9547537d608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bff62bfcb0946eeac42ebd190731351, user_id=9212efdd0bc840248786839746836571, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.355151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a173fd-57a1-4bdd-90da-e9547537d608 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.369658 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1275/2551] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:02 2022] PUT /identity/v3/projects/1bff62bfcb0946eeac42ebd190731351/users/9212efdd0bc840248786839746836571/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:02.381223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef8c46-ed63-4415-b1d6-e0bf05506918 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.381527 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef8c46-ed63-4415-b1d6-e0bf05506918 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.381809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef8c46-ed63-4415-b1d6-e0bf05506918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.414073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-94ef8c46-ed63-4415-b1d6-e0bf05506918 None None] MFA Rules not processed for user `9212efdd0bc840248786839746836571`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:02.415358 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-94ef8c46-ed63-4415-b1d6-e0bf05506918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:02.524147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1277/2552] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:02.530542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a173fd-57a1-4bdd-90da-e9547537d608 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.587345 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:02.628285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a173fd-57a1-4bdd-90da-e9547537d608 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.634181 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3af7c0-bdb2-4aac-b76a-fc16f2fbbad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.634945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3af7c0-bdb2-4aac-b76a-fc16f2fbbad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.635318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3af7c0-bdb2-4aac-b76a-fc16f2fbbad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.635679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3af7c0-bdb2-4aac-b76a-fc16f2fbbad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.636482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3af7c0-bdb2-4aac-b76a-fc16f2fbbad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.639534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3af7c0-bdb2-4aac-b76a-fc16f2fbbad8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.662613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef8c46-ed63-4415-b1d6-e0bf05506918 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.664515 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-94ef8c46-ed63-4415-b1d6-e0bf05506918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:02.673734 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1276/2553] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:02 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:02.682364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3af7c0-bdb2-4aac-b76a-fc16f2fbbad8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.689565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef8c46-ed63-4415-b1d6-e0bf05506918 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.719144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:02.774213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3af7c0-bdb2-4aac-b76a-fc16f2fbbad8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.780165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc07eae-0e6d-4ac0-8a2d-05e4450908b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.781123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc07eae-0e6d-4ac0-8a2d-05e4450908b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.781788 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc07eae-0e6d-4ac0-8a2d-05e4450908b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.782359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc07eae-0e6d-4ac0-8a2d-05e4450908b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.783293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc07eae-0e6d-4ac0-8a2d-05e4450908b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.784877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.785748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.786043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.786178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc07eae-0e6d-4ac0-8a2d-05e4450908b4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.786387 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] PATH_INFO: `/v3/projects/c6b4e932b06543e5ae9f8e125a3f5b53` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.787585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b4e932b06543e5ae9f8e125a3f5b53)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.789499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.811337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1277/2554] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:02 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:02.813488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc07eae-0e6d-4ac0-8a2d-05e4450908b4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.824029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc07eae-0e6d-4ac0-8a2d-05e4450908b4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.829705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec32a-7059-478b-af37-19ecc99af7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.830479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec32a-7059-478b-af37-19ecc99af7fb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.830765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec32a-7059-478b-af37-19ecc99af7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.831050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec32a-7059-478b-af37-19ecc99af7fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.833152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aec32a-7059-478b-af37-19ecc99af7fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.833627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aec32a-7059-478b-af37-19ecc99af7fb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.845333 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1278/2555] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:02.849901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec32a-7059-478b-af37-19ecc99af7fb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.854645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b4e932b06543e5ae9f8e125a3f5b53', 'request_id': 'req-c9e856fb-a7e3-492a-8e22-c48469c7ded1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.860008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec32a-7059-478b-af37-19ecc99af7fb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.864491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b4e932b06543e5ae9f8e125a3f5b53', 'request_id': 'req-c9e856fb-a7e3-492a-8e22-c48469c7ded1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.871565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b4e932b06543e5ae9f8e125a3f5b53', 'request_id': 'req-c9e856fb-a7e3-492a-8e22-c48469c7ded1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.887536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] The token cache is being invalidate because project c6b4e932b06543e5ae9f8e125a3f5b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:02.887844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.888508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:02.890363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1278/2556] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:02 2022] DELETE /identity/v3/projects/c6b4e932b06543e5ae9f8e125a3f5b53 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:02.895497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.897241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:02.934684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10117ffc-b800-4482-b2cc-7d1dc6339ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.936522 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10117ffc-b800-4482-b2cc-7d1dc6339ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.936690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10117ffc-b800-4482-b2cc-7d1dc6339ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.936989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10117ffc-b800-4482-b2cc-7d1dc6339ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.937779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10117ffc-b800-4482-b2cc-7d1dc6339ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.939996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10117ffc-b800-4482-b2cc-7d1dc6339ec3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.951401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1279/2557] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:02.955745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10117ffc-b800-4482-b2cc-7d1dc6339ec3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:02.957592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-10117ffc-b800-4482-b2cc-7d1dc6339ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:02.971897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10117ffc-b800-4482-b2cc-7d1dc6339ec3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.977243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29a5f5-adf1-492f-9c55-7896d20bf930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.978187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29a5f5-adf1-492f-9c55-7896d20bf930 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.978476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29a5f5-adf1-492f-9c55-7896d20bf930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.978763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29a5f5-adf1-492f-9c55-7896d20bf930 admin admin] PATH_INFO: `/v3/projects/49492563f97a4b4490e57278513a3517/users/9b60b3d4a506401fa5db85dee75d6fca/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.990591 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e856fb-a7e3-492a-8e22-c48469c7ded1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:02.995650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29a5f5-adf1-492f-9c55-7896d20bf930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49492563f97a4b4490e57278513a3517, user_id=9b60b3d4a506401fa5db85dee75d6fca, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:02.996648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:02.997809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:02.998167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:02.998814 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] PATH_INFO: `/v3/users/d15e8e272a474924acc845f8088449a2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:02.999190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29a5f5-adf1-492f-9c55-7896d20bf930 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:02.999961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15e8e272a474924acc845f8088449a2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.002549 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.015674 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1280/2558] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:02 2022] PUT /identity/v3/projects/49492563f97a4b4490e57278513a3517/users/9b60b3d4a506401fa5db85dee75d6fca/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:03.026450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f5d15-fdcc-4147-886a-177e90412301 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.026799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f5d15-fdcc-4147-886a-177e90412301 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.027247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f5d15-fdcc-4147-886a-177e90412301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.058883 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b92f5d15-fdcc-4147-886a-177e90412301 None None] MFA Rules not processed for user `9b60b3d4a506401fa5db85dee75d6fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:03.059527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b92f5d15-fdcc-4147-886a-177e90412301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:03.065529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:03.075713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15e8e272a474924acc845f8088449a2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.076523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15e8e272a474924acc845f8088449a2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.080524 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:03.087578 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15e8e272a474924acc845f8088449a2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.104204 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15e8e272a474924acc845f8088449a2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.112386 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15e8e272a474924acc845f8088449a2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.119280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1279/2559] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:02 2022] DELETE /identity/v3/users/d15e8e272a474924acc845f8088449a2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:03.120189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.121545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:03.142386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1281/2560] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:03.149766 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f5d15-fdcc-4147-886a-177e90412301 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.152477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b92f5d15-fdcc-4147-886a-177e90412301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:03.224173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a2a06-c7a3-4c00-8b9f-bd45c6045d7e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.228027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2bd0b-ca73-4e0e-b6eb-f5453f5a6853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:03.228732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2bd0b-ca73-4e0e-b6eb-f5453f5a6853 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.229065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2bd0b-ca73-4e0e-b6eb-f5453f5a6853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.229325 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2bd0b-ca73-4e0e-b6eb-f5453f5a6853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.231302 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89c2bd0b-ca73-4e0e-b6eb-f5453f5a6853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:03.288192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f5d15-fdcc-4147-886a-177e90412301 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.312376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2bd0b-ca73-4e0e-b6eb-f5453f5a6853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.314664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2bd0b-ca73-4e0e-b6eb-f5453f5a6853 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.335345 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1280/2561] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:03.368811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350e87-52cf-4c51-83ea-3986a1764bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:03.369495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350e87-52cf-4c51-83ea-3986a1764bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.369789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350e87-52cf-4c51-83ea-3986a1764bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.370068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350e87-52cf-4c51-83ea-3986a1764bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.370767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03350e87-52cf-4c51-83ea-3986a1764bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.373164 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03350e87-52cf-4c51-83ea-3986a1764bb5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.409390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1282/2562] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:03 2022] POST /identity/v3/projects => generated 379 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:03.410673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2bd0b-ca73-4e0e-b6eb-f5453f5a6853 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.421555 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2bd0b-ca73-4e0e-b6eb-f5453f5a6853 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.488496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eedb29-8d5c-494a-84ac-b74a2491a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:03.489093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eedb29-8d5c-494a-84ac-b74a2491a6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.489649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eedb29-8d5c-494a-84ac-b74a2491a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.489933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eedb29-8d5c-494a-84ac-b74a2491a6ee admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.490633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eedb29-8d5c-494a-84ac-b74a2491a6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.492595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eedb29-8d5c-494a-84ac-b74a2491a6ee admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.516043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1281/2563] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:03 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:03.520293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350e87-52cf-4c51-83ea-3986a1764bb5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.520893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eedb29-8d5c-494a-84ac-b74a2491a6ee admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.522730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3eedb29-8d5c-494a-84ac-b74a2491a6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:03.533013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350e87-52cf-4c51-83ea-3986a1764bb5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.534952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eedb29-8d5c-494a-84ac-b74a2491a6ee admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.540504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376cf34-f63c-4684-9c50-19951aaabaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:03.541904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:03.542492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376cf34-f63c-4684-9c50-19951aaabaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.542602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.542695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.543150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376cf34-f63c-4684-9c50-19951aaabaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.543472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376cf34-f63c-4684-9c50-19951aaabaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.543744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] PATH_INFO: `/v3/projects/157674d652dd4236b571b2b0f78fd956` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.545048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] RBAC: Authorizing `identity:delete_project(project_id=157674d652dd4236b571b2b0f78fd956)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.545371 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376cf34-f63c-4684-9c50-19951aaabaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.547018 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.547143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376cf34-f63c-4684-9c50-19951aaabaaa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.550103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:03.572850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1283/2564] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:03.576873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376cf34-f63c-4684-9c50-19951aaabaaa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.589929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376cf34-f63c-4684-9c50-19951aaabaaa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.601926 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84f060-6a5c-4b54-8816-122a6575d370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:03.603064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84f060-6a5c-4b54-8816-122a6575d370 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.603559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84f060-6a5c-4b54-8816-122a6575d370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.603846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84f060-6a5c-4b54-8816-122a6575d370 admin admin] PATH_INFO: `/v3/projects/fb7338574a1b42ccb3da38fc62886dfb/users/41f2616ef76940889f148dda3b403e06/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.620985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157674d652dd4236b571b2b0f78fd956', 'request_id': 'req-50faa2be-6b58-41ab-baa5-6c25c933332e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.621557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84f060-6a5c-4b54-8816-122a6575d370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb7338574a1b42ccb3da38fc62886dfb, user_id=41f2616ef76940889f148dda3b403e06, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.625298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84f060-6a5c-4b54-8816-122a6575d370 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.628054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157674d652dd4236b571b2b0f78fd956', 'request_id': 'req-50faa2be-6b58-41ab-baa5-6c25c933332e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.637005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157674d652dd4236b571b2b0f78fd956', 'request_id': 'req-50faa2be-6b58-41ab-baa5-6c25c933332e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.642127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1284/2565] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:03 2022] PUT /identity/v3/projects/fb7338574a1b42ccb3da38fc62886dfb/users/41f2616ef76940889f148dda3b403e06/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:03.646753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84f060-6a5c-4b54-8816-122a6575d370 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.653262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] The token cache is being invalidate because project 157674d652dd4236b571b2b0f78fd956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:03.653438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:03.653564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.654568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-50faa2be-6b58-41ab-baa5-6c25c933332e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:03.657360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1282/2566] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:03 2022] DELETE /identity/v3/projects/157674d652dd4236b571b2b0f78fd956 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:03.663305 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68386034-536f-4eac-9220-787cd13439b8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.663961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68386034-536f-4eac-9220-787cd13439b8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.663961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68386034-536f-4eac-9220-787cd13439b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.685923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-68386034-536f-4eac-9220-787cd13439b8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:03.687463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-68386034-536f-4eac-9220-787cd13439b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:03.729743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84f060-6a5c-4b54-8816-122a6575d370 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.735928 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1a01a-c652-4309-861e-f8b44a5e9b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:03.737107 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1a01a-c652-4309-861e-f8b44a5e9b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.737107 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1a01a-c652-4309-861e-f8b44a5e9b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.737357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1a01a-c652-4309-861e-f8b44a5e9b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.738074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1a01a-c652-4309-861e-f8b44a5e9b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.740158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1a01a-c652-4309-861e-f8b44a5e9b4d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.752099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1285/2567] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:03.756457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1a01a-c652-4309-861e-f8b44a5e9b4d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.759024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-52a1a01a-c652-4309-861e-f8b44a5e9b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:03.774266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1a01a-c652-4309-861e-f8b44a5e9b4d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.780008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04001a-fbb8-4b80-ab27-7f1b0d7fd82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:03.780919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04001a-fbb8-4b80-ab27-7f1b0d7fd82b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.781207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04001a-fbb8-4b80-ab27-7f1b0d7fd82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.781392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1283/2568] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:03.782197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04001a-fbb8-4b80-ab27-7f1b0d7fd82b admin admin] PATH_INFO: `/v3/projects/fb7338574a1b42ccb3da38fc62886dfb/users/41f2616ef76940889f148dda3b403e06/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.784912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04001a-fbb8-4b80-ab27-7f1b0d7fd82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb7338574a1b42ccb3da38fc62886dfb, user_id=41f2616ef76940889f148dda3b403e06, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.787641 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04001a-fbb8-4b80-ab27-7f1b0d7fd82b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.790179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a2e8-f900-4783-a088-b244c5e9c68f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.790504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a2e8-f900-4783-a088-b244c5e9c68f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.790888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a2e8-f900-4783-a088-b244c5e9c68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.800894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1286/2569] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:03 2022] PUT /identity/v3/projects/fb7338574a1b42ccb3da38fc62886dfb/users/41f2616ef76940889f148dda3b403e06/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:03.812388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfa77d-7eaf-4e03-b3ca-9ad35d0d749b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.812696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfa77d-7eaf-4e03-b3ca-9ad35d0d749b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.813000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfa77d-7eaf-4e03-b3ca-9ad35d0d749b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.814790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0994a2e8-f900-4783-a088-b244c5e9c68f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:03.816175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0994a2e8-f900-4783-a088-b244c5e9c68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:03.861668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b9bfa77d-7eaf-4e03-b3ca-9ad35d0d749b None None] MFA Rules not processed for user `41f2616ef76940889f148dda3b403e06`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:03.864105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b9bfa77d-7eaf-4e03-b3ca-9ad35d0d749b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:03.898190 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1284/2570] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:03.902280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a2e8-f900-4783-a088-b244c5e9c68f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.919802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a2e8-f900-4783-a088-b244c5e9c68f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.926602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383df450-d110-46dc-bb2e-ae1382812c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:03.927315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df450-d110-46dc-bb2e-ae1382812c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:03.927716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df450-d110-46dc-bb2e-ae1382812c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:03.928008 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df450-d110-46dc-bb2e-ae1382812c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:03.928751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383df450-d110-46dc-bb2e-ae1382812c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:22:03.929250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383df450-d110-46dc-bb2e-ae1382812c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:03.931402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383df450-d110-46dc-bb2e-ae1382812c12 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:03.947105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1285/2571] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:22:03.952414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383df450-d110-46dc-bb2e-ae1382812c12 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:03.980267 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1287/2572] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:03.983660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383df450-d110-46dc-bb2e-ae1382812c12 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:03.998592 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfa77d-7eaf-4e03-b3ca-9ad35d0d749b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.000093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b9bfa77d-7eaf-4e03-b3ca-9ad35d0d749b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:04.001940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ac33d-594d-4f14-8270-e5a5c1394cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.002632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ac33d-594d-4f14-8270-e5a5c1394cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.002926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ac33d-594d-4f14-8270-e5a5c1394cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.003611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ac33d-594d-4f14-8270-e5a5c1394cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.004424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ac33d-594d-4f14-8270-e5a5c1394cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.006422 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ac33d-594d-4f14-8270-e5a5c1394cfc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.028090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:04.049740 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1286/2573] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:03 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:04.053653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ac33d-594d-4f14-8270-e5a5c1394cfc admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.089807 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfa77d-7eaf-4e03-b3ca-9ad35d0d749b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.130996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:04.137820 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ac33d-594d-4f14-8270-e5a5c1394cfc admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.145574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e3413-3536-4577-a72a-87dfeba2c99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.146272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e3413-3536-4577-a72a-87dfeba2c99b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.146573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e3413-3536-4577-a72a-87dfeba2c99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.146882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e3413-3536-4577-a72a-87dfeba2c99b admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.147751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e3413-3536-4577-a72a-87dfeba2c99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.149741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e3413-3536-4577-a72a-87dfeba2c99b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.174629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1287/2574] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:04 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:04.174843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c47de-a0b8-4354-83a5-ee5e1e3887f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.175576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c47de-a0b8-4354-83a5-ee5e1e3887f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.175868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c47de-a0b8-4354-83a5-ee5e1e3887f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.176153 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c47de-a0b8-4354-83a5-ee5e1e3887f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.179628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e3413-3536-4577-a72a-87dfeba2c99b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.191543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e3413-3536-4577-a72a-87dfeba2c99b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.197297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f37b4a-fc5a-41f4-92a1-0d96decd47ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.198093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f37b4a-fc5a-41f4-92a1-0d96decd47ae admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.198446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f37b4a-fc5a-41f4-92a1-0d96decd47ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.198893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f37b4a-fc5a-41f4-92a1-0d96decd47ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.199979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f37b4a-fc5a-41f4-92a1-0d96decd47ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.201717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f37b4a-fc5a-41f4-92a1-0d96decd47ae admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.217701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1288/2575] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:04.218908 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f37b4a-fc5a-41f4-92a1-0d96decd47ae admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.231465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f37b4a-fc5a-41f4-92a1-0d96decd47ae admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.236576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bee0da-b3fa-4c78-977c-073799338bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.237519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bee0da-b3fa-4c78-977c-073799338bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.237690 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bee0da-b3fa-4c78-977c-073799338bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.238017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bee0da-b3fa-4c78-977c-073799338bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.238758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bee0da-b3fa-4c78-977c-073799338bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.240884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bee0da-b3fa-4c78-977c-073799338bc8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.263128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1289/2576] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:04.267796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bee0da-b3fa-4c78-977c-073799338bc8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.268049 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c47de-a0b8-4354-83a5-ee5e1e3887f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.270855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c47de-a0b8-4354-83a5-ee5e1e3887f6 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.278801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bee0da-b3fa-4c78-977c-073799338bc8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.284175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7b5a3-c382-45e6-aebc-c95928888281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.285076 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b5a3-c382-45e6-aebc-c95928888281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.285368 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b5a3-c382-45e6-aebc-c95928888281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.285647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b5a3-c382-45e6-aebc-c95928888281 admin admin] PATH_INFO: `/v3/projects/504ab1a4133c4ca7b44b77300ecea2ec/users/0bf75458ad704a2bb0ca729ecdc3d498/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.292121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1288/2577] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:04.307577 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7b5a3-c382-45e6-aebc-c95928888281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504ab1a4133c4ca7b44b77300ecea2ec, user_id=0bf75458ad704a2bb0ca729ecdc3d498, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.313462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7b5a3-c382-45e6-aebc-c95928888281 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.340727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1290/2578] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:04 2022] PUT /identity/v3/projects/504ab1a4133c4ca7b44b77300ecea2ec/users/0bf75458ad704a2bb0ca729ecdc3d498/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:04.352832 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7b5a3-c382-45e6-aebc-c95928888281 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.353541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dc10e-ea3f-4f8a-aabc-99568f9bd3d1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.354364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dc10e-ea3f-4f8a-aabc-99568f9bd3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.354716 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dc10e-ea3f-4f8a-aabc-99568f9bd3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.355224 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3bd7b5a3-c382-45e6-aebc-c95928888281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:04.398753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-357dc10e-ea3f-4f8a-aabc-99568f9bd3d1 None None] MFA Rules not processed for user `0bf75458ad704a2bb0ca729ecdc3d498`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:04.400050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-357dc10e-ea3f-4f8a-aabc-99568f9bd3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:04.441955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7b5a3-c382-45e6-aebc-c95928888281 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.446403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776090fb-24f5-4d5c-b52c-6f9d727a5f27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.447071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776090fb-24f5-4d5c-b52c-6f9d727a5f27 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.447425 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776090fb-24f5-4d5c-b52c-6f9d727a5f27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.447966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776090fb-24f5-4d5c-b52c-6f9d727a5f27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.449706 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-776090fb-24f5-4d5c-b52c-6f9d727a5f27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:04.477416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1289/2579] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:04.482044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dc10e-ea3f-4f8a-aabc-99568f9bd3d1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.514777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:04.520267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776090fb-24f5-4d5c-b52c-6f9d727a5f27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.522681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776090fb-24f5-4d5c-b52c-6f9d727a5f27 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.533384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1291/2580] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:04.552550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dc10e-ea3f-4f8a-aabc-99568f9bd3d1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.562129 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97414c-65b9-4fee-9e71-1c948c2d9ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.562827 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97414c-65b9-4fee-9e71-1c948c2d9ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.563799 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97414c-65b9-4fee-9e71-1c948c2d9ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.564093 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97414c-65b9-4fee-9e71-1c948c2d9ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.564798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97414c-65b9-4fee-9e71-1c948c2d9ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.566754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97414c-65b9-4fee-9e71-1c948c2d9ef1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.582535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:04.597656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1290/2581] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:04 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:04.602195 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776090fb-24f5-4d5c-b52c-6f9d727a5f27 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.614123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776090fb-24f5-4d5c-b52c-6f9d727a5f27 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.681187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f09e47-fc48-42f9-9c28-33f8718704ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.681980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f09e47-fc48-42f9-9c28-33f8718704ea admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.682338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f09e47-fc48-42f9-9c28-33f8718704ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.682702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f09e47-fc48-42f9-9c28-33f8718704ea admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.683548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f09e47-fc48-42f9-9c28-33f8718704ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.685666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f09e47-fc48-42f9-9c28-33f8718704ea admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.714743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1292/2582] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:04 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:04.715615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97414c-65b9-4fee-9e71-1c948c2d9ef1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.726737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97414c-65b9-4fee-9e71-1c948c2d9ef1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.732794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db11b8c-ddda-4c7c-b601-fde16007fdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.734919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db11b8c-ddda-4c7c-b601-fde16007fdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.734919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db11b8c-ddda-4c7c-b601-fde16007fdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.734919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db11b8c-ddda-4c7c-b601-fde16007fdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.736604 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db11b8c-ddda-4c7c-b601-fde16007fdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.740019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db11b8c-ddda-4c7c-b601-fde16007fdf5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.765021 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1291/2583] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:04.767598 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f09e47-fc48-42f9-9c28-33f8718704ea admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.786311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f09e47-fc48-42f9-9c28-33f8718704ea admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.795298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906f72-1f50-43bf-b6ad-1da27504152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.796457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906f72-1f50-43bf-b6ad-1da27504152f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.797032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906f72-1f50-43bf-b6ad-1da27504152f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.798110 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906f72-1f50-43bf-b6ad-1da27504152f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.799526 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3906f72-1f50-43bf-b6ad-1da27504152f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.802601 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3906f72-1f50-43bf-b6ad-1da27504152f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.824938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1293/2584] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:04.826931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db11b8c-ddda-4c7c-b601-fde16007fdf5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:04.838676 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db11b8c-ddda-4c7c-b601-fde16007fdf5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:04.843424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b224af1-2a95-46b0-a3c7-39fb0dc8554f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:04.844288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b224af1-2a95-46b0-a3c7-39fb0dc8554f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.844581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b224af1-2a95-46b0-a3c7-39fb0dc8554f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.844866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b224af1-2a95-46b0-a3c7-39fb0dc8554f admin admin] PATH_INFO: `/v3/projects/2d21b6f50e584af18aa71e957b001fcf/users/c87704290bde424996fb99b8af7ae80f/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:04.863786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b224af1-2a95-46b0-a3c7-39fb0dc8554f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d21b6f50e584af18aa71e957b001fcf, user_id=c87704290bde424996fb99b8af7ae80f, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:04.867564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b224af1-2a95-46b0-a3c7-39fb0dc8554f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:04.896909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1292/2585] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:04 2022] PUT /identity/v3/projects/2d21b6f50e584af18aa71e957b001fcf/users/c87704290bde424996fb99b8af7ae80f/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:04.993494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bfaad-bd50-4f6b-81f6-81bd9e887588 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:04.993766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bfaad-bd50-4f6b-81f6-81bd9e887588 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:04.994047 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bfaad-bd50-4f6b-81f6-81bd9e887588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:05.038279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f28bfaad-bd50-4f6b-81f6-81bd9e887588 None None] MFA Rules not processed for user `c87704290bde424996fb99b8af7ae80f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:05.039788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f28bfaad-bd50-4f6b-81f6-81bd9e887588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:05.126660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1294/2586] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:05.131867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b224af1-2a95-46b0-a3c7-39fb0dc8554f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:05.229215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b224af1-2a95-46b0-a3c7-39fb0dc8554f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:05.241929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b5410-fa39-4d55-b768-4a93a805a0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:05.243231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b5410-fa39-4d55-b768-4a93a805a0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:05.243887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b5410-fa39-4d55-b768-4a93a805a0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:05.244470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b5410-fa39-4d55-b768-4a93a805a0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:05.245954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b5410-fa39-4d55-b768-4a93a805a0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:05.250173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b5410-fa39-4d55-b768-4a93a805a0b4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:05.298239 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1293/2587] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:05 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:05.299547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bfaad-bd50-4f6b-81f6-81bd9e887588 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:05.318711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bfaad-bd50-4f6b-81f6-81bd9e887588 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:05.433014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f4f6c-603f-441c-9fe0-dd9b760fe07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:05.433793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f4f6c-603f-441c-9fe0-dd9b760fe07f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:05.434153 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f4f6c-603f-441c-9fe0-dd9b760fe07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:05.434500 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f4f6c-603f-441c-9fe0-dd9b760fe07f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:05.435320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f4f6c-603f-441c-9fe0-dd9b760fe07f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:05.437339 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f4f6c-603f-441c-9fe0-dd9b760fe07f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:05.445940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:05.465137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1295/2588] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:05 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:05.468123 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b5410-fa39-4d55-b768-4a93a805a0b4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:05.480454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b5410-fa39-4d55-b768-4a93a805a0b4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:05.489149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a68aa9-1394-495c-9112-644d3fdabeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:05.490606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a68aa9-1394-495c-9112-644d3fdabeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:05.491106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a68aa9-1394-495c-9112-644d3fdabeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:05.491671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a68aa9-1394-495c-9112-644d3fdabeb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:05.493191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a68aa9-1394-495c-9112-644d3fdabeb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:05.496325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a68aa9-1394-495c-9112-644d3fdabeb4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:05.514790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1294/2589] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:05.519443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f4f6c-603f-441c-9fe0-dd9b760fe07f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:05.534834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f4f6c-603f-441c-9fe0-dd9b760fe07f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:05.539774 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456119b0-127f-4f8a-aaec-d576df9003bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:05.540700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456119b0-127f-4f8a-aaec-d576df9003bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:05.541613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456119b0-127f-4f8a-aaec-d576df9003bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:05.541613 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456119b0-127f-4f8a-aaec-d576df9003bf admin admin] PATH_INFO: `/v3/projects/9b730b8aa59c4e148409aaa63daa9292/users/a5d70ca2f58049f18eb7edd6d2ee124b/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:05.560166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456119b0-127f-4f8a-aaec-d576df9003bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b730b8aa59c4e148409aaa63daa9292, user_id=a5d70ca2f58049f18eb7edd6d2ee124b, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:05.562904 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456119b0-127f-4f8a-aaec-d576df9003bf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:05.578299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1296/2590] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:05 2022] PUT /identity/v3/projects/9b730b8aa59c4e148409aaa63daa9292/users/a5d70ca2f58049f18eb7edd6d2ee124b/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:05.582956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a68aa9-1394-495c-9112-644d3fdabeb4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:05.603248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a68aa9-1394-495c-9112-644d3fdabeb4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:05.684800 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b67f73-5642-488a-ae08-f4b2f99d2422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:05.686167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b67f73-5642-488a-ae08-f4b2f99d2422 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:05.687030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b67f73-5642-488a-ae08-f4b2f99d2422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:05.687460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b67f73-5642-488a-ae08-f4b2f99d2422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:05.688494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b67f73-5642-488a-ae08-f4b2f99d2422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:05.690229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b67f73-5642-488a-ae08-f4b2f99d2422 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:05.708431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1295/2591] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:05.708698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456119b0-127f-4f8a-aaec-d576df9003bf admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:05.721514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456119b0-127f-4f8a-aaec-d576df9003bf admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:05.733822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d52d2-a340-4b6a-9a1b-8da523aa6915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:05.734713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d52d2-a340-4b6a-9a1b-8da523aa6915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:05.735032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d52d2-a340-4b6a-9a1b-8da523aa6915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:05.735382 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d52d2-a340-4b6a-9a1b-8da523aa6915 admin admin] PATH_INFO: `/v3/projects/9b730b8aa59c4e148409aaa63daa9292/users/a5d70ca2f58049f18eb7edd6d2ee124b/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:05.739010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d52d2-a340-4b6a-9a1b-8da523aa6915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b730b8aa59c4e148409aaa63daa9292, user_id=a5d70ca2f58049f18eb7edd6d2ee124b, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:05.741072 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d52d2-a340-4b6a-9a1b-8da523aa6915 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:05.767794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1297/2592] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:05 2022] PUT /identity/v3/projects/9b730b8aa59c4e148409aaa63daa9292/users/a5d70ca2f58049f18eb7edd6d2ee124b/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:05.770282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f310-14de-4b7a-86e1-86e02ca6618a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:05.770853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f310-14de-4b7a-86e1-86e02ca6618a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:05.771293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f310-14de-4b7a-86e1-86e02ca6618a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:05.802791 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b923f310-14de-4b7a-86e1-86e02ca6618a None None] MFA Rules not processed for user `a5d70ca2f58049f18eb7edd6d2ee124b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:05.804425 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b923f310-14de-4b7a-86e1-86e02ca6618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:05.825459 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:05.903985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1296/2593] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:05.916837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d52d2-a340-4b6a-9a1b-8da523aa6915 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:05.993429 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d52d2-a340-4b6a-9a1b-8da523aa6915 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:05.997623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e201fe-925f-436d-9fb6-f24fc08a06e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:05.998461 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e201fe-925f-436d-9fb6-f24fc08a06e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:05.998748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e201fe-925f-436d-9fb6-f24fc08a06e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:05.999037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e201fe-925f-436d-9fb6-f24fc08a06e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:06.089464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e201fe-925f-436d-9fb6-f24fc08a06e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:06.090937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e201fe-925f-436d-9fb6-f24fc08a06e4 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:06.117294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1298/2594] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:07.695563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f310-14de-4b7a-86e1-86e02ca6618a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:07.773695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f310-14de-4b7a-86e1-86e02ca6618a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:07.778437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:07.779430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:07.779718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:07.779995 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] PATH_INFO: `/v3/users/9212efdd0bc840248786839746836571` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:07.781186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9212efdd0bc840248786839746836571)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:07.783884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:07.835228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9212efdd0bc840248786839746836571'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:07.844102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e201fe-925f-436d-9fb6-f24fc08a06e4 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:07.845326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9212efdd0bc840248786839746836571'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:07.853364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9212efdd0bc840248786839746836571'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:07.856943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e201fe-925f-436d-9fb6-f24fc08a06e4 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:07.862837 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ccc57-93b7-4b9d-86e5-e75e34bcb77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:07.863979 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ccc57-93b7-4b9d-86e5-e75e34bcb77c service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:07.864743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ccc57-93b7-4b9d-86e5-e75e34bcb77c service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:07.865271 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ccc57-93b7-4b9d-86e5-e75e34bcb77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:07.873885 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9212efdd0bc840248786839746836571'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:07.874877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8561d0e1-5e53-4c72-a8b3-18ee8ea96555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9212efdd0bc840248786839746836571'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:07.883543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1297/2595] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:07 2022] DELETE /identity/v3/users/9212efdd0bc840248786839746836571 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:07.890286 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d51b28-99ce-4f26-805d-c4ec378a5176 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:07.890720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d51b28-99ce-4f26-805d-c4ec378a5176 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:07.891695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d51b28-99ce-4f26-805d-c4ec378a5176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:07.917160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b6d51b28-99ce-4f26-805d-c4ec378a5176 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:07.918372 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b6d51b28-99ce-4f26-805d-c4ec378a5176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:07.938818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ccc57-93b7-4b9d-86e5-e75e34bcb77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:07.941293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ccc57-93b7-4b9d-86e5-e75e34bcb77c service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:07.947137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:07.950432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1299/2596] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:07.981472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1298/2597] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:07.992469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ccc57-93b7-4b9d-86e5-e75e34bcb77c service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:07.993886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d26ccc57-93b7-4b9d-86e5-e75e34bcb77c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:08.003843 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:08.054645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ccc57-93b7-4b9d-86e5-e75e34bcb77c service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:08.106262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b35fd6-5929-4be6-953b-ab8c3feacbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:08.106986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b35fd6-5929-4be6-953b-ab8c3feacbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:08.107415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b35fd6-5929-4be6-953b-ab8c3feacbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:08.107702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b35fd6-5929-4be6-953b-ab8c3feacbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:08.117217 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b35fd6-5929-4be6-953b-ab8c3feacbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:08.118691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b35fd6-5929-4be6-953b-ab8c3feacbf5 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:08.128791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1300/2598] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:08.431349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d51b28-99ce-4f26-805d-c4ec378a5176 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:08.433801 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b6d51b28-99ce-4f26-805d-c4ec378a5176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:08.464770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d51b28-99ce-4f26-805d-c4ec378a5176 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:08.473094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:08.474722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:08.475190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:08.475838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] PATH_INFO: `/v3/projects/1bff62bfcb0946eeac42ebd190731351` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:08.478215 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bff62bfcb0946eeac42ebd190731351)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:08.481857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:08.532101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b35fd6-5929-4be6-953b-ab8c3feacbf5 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:08.533638 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a0b35fd6-5929-4be6-953b-ab8c3feacbf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:08.560476 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bff62bfcb0946eeac42ebd190731351', 'request_id': 'req-57dfd3ff-6909-4be4-99e7-c709d5523c9e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.572131 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bff62bfcb0946eeac42ebd190731351', 'request_id': 'req-57dfd3ff-6909-4be4-99e7-c709d5523c9e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.580050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bff62bfcb0946eeac42ebd190731351', 'request_id': 'req-57dfd3ff-6909-4be4-99e7-c709d5523c9e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.595930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] The token cache is being invalidate because project 1bff62bfcb0946eeac42ebd190731351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:08.596611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.597778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.601133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b35fd6-5929-4be6-953b-ab8c3feacbf5 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:08.603338 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1299/2599] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:08 2022] DELETE /identity/v3/projects/1bff62bfcb0946eeac42ebd190731351 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:08.606772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:08.608236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:08.662859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3feea6-07c8-49e5-be5b-e326dd14eeee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:08.663470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feea6-07c8-49e5-be5b-e326dd14eeee service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:08.663770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feea6-07c8-49e5-be5b-e326dd14eeee service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:08.664051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feea6-07c8-49e5-be5b-e326dd14eeee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:08.666259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8a3feea6-07c8-49e5-be5b-e326dd14eeee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:08.697466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfd3ff-6909-4be4-99e7-c709d5523c9e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:08.705554 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:08.706019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:08.707034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:08.707562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:08.708077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] PATH_INFO: `/v3/users/9b60b3d4a506401fa5db85dee75d6fca` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:08.710108 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b60b3d4a506401fa5db85dee75d6fca)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:08.713823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:08.735794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3feea6-07c8-49e5-be5b-e326dd14eeee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:08.737236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3feea6-07c8-49e5-be5b-e326dd14eeee service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:08.748620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1301/2600] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:08.790811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b60b3d4a506401fa5db85dee75d6fca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.799094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b60b3d4a506401fa5db85dee75d6fca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.808705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:08.819012 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b60b3d4a506401fa5db85dee75d6fca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.847136 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b60b3d4a506401fa5db85dee75d6fca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.848159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b60b3d4a506401fa5db85dee75d6fca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:08.856733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1300/2601] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:08 2022] DELETE /identity/v3/users/9b60b3d4a506401fa5db85dee75d6fca => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:09.184067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3feea6-07c8-49e5-be5b-e326dd14eeee service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:09.195553 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3feea6-07c8-49e5-be5b-e326dd14eeee service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:09.253715 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:09.254557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:09.254842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:09.255119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] PATH_INFO: `/v3/projects/49492563f97a4b4490e57278513a3517` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:09.256353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49492563f97a4b4490e57278513a3517)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:09.258286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:09.304333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49492563f97a4b4490e57278513a3517', 'request_id': 'req-6b9f1140-1a9c-4e68-bcc1-71a962f60875'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.309990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49492563f97a4b4490e57278513a3517', 'request_id': 'req-6b9f1140-1a9c-4e68-bcc1-71a962f60875'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.317726 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49492563f97a4b4490e57278513a3517', 'request_id': 'req-6b9f1140-1a9c-4e68-bcc1-71a962f60875'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.329890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] The token cache is being invalidate because project 49492563f97a4b4490e57278513a3517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:09.330185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.330862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.332444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1302/2602] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:09 2022] DELETE /identity/v3/projects/49492563f97a4b4490e57278513a3517 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:09.336483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:09.337892 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:09.350722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d90c62-546e-4eba-b45b-39f545657fa5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:09.407527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:09.408409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:09.408698 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:09.408978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] PATH_INFO: `/v3/users/41f2616ef76940889f148dda3b403e06` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:09.410192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f2616ef76940889f148dda3b403e06)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:09.412175 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:09.472606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f2616ef76940889f148dda3b403e06'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.481619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f2616ef76940889f148dda3b403e06'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.490353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f2616ef76940889f148dda3b403e06'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.502518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f2616ef76940889f148dda3b403e06'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.503170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7c079800-a474-4a99-90ab-fa0e1e734709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f2616ef76940889f148dda3b403e06'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.509068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1301/2603] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:09 2022] DELETE /identity/v3/users/41f2616ef76940889f148dda3b403e06 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:09.792957 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:09.794476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:09.808136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f1140-1a9c-4e68-bcc1-71a962f60875 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:09.869444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:09.870303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:09.870587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:09.870862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] PATH_INFO: `/v3/projects/fb7338574a1b42ccb3da38fc62886dfb` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:09.872083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb7338574a1b42ccb3da38fc62886dfb)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:09.873983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:09.876269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:09.927407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb7338574a1b42ccb3da38fc62886dfb', 'request_id': 'req-bfba48db-8a46-46c0-acc1-27fac820ae9a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.933914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb7338574a1b42ccb3da38fc62886dfb', 'request_id': 'req-bfba48db-8a46-46c0-acc1-27fac820ae9a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.941243 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb7338574a1b42ccb3da38fc62886dfb', 'request_id': 'req-bfba48db-8a46-46c0-acc1-27fac820ae9a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.955895 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] The token cache is being invalidate because project fb7338574a1b42ccb3da38fc62886dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:09.956185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.956847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfba48db-8a46-46c0-acc1-27fac820ae9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:09.958741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1303/2604] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:09 2022] DELETE /identity/v3/projects/fb7338574a1b42ccb3da38fc62886dfb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:09.964763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cce89c-c982-4c36-8ce5-7329d7993c4f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:09.965216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cce89c-c982-4c36-8ce5-7329d7993c4f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:09.965340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cce89c-c982-4c36-8ce5-7329d7993c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:09.983441 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e8cce89c-c982-4c36-8ce5-7329d7993c4f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:09.985236 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e8cce89c-c982-4c36-8ce5-7329d7993c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:10.037601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:10.051821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1302/2605] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:10.060259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8e429-7f9a-42ed-a740-ccd33eb5dbb4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:10.060556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8e429-7f9a-42ed-a740-ccd33eb5dbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:10.060842 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8e429-7f9a-42ed-a740-ccd33eb5dbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:10.080727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7fe8e429-7f9a-42ed-a740-ccd33eb5dbb4 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:10.082024 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7fe8e429-7f9a-42ed-a740-ccd33eb5dbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:10.090464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1304/2606] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:10.095446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cce89c-c982-4c36-8ce5-7329d7993c4f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:10.106045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cce89c-c982-4c36-8ce5-7329d7993c4f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:10.111960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247fed6-3c03-4718-9666-bd3c2e5d12f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:10.112832 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247fed6-3c03-4718-9666-bd3c2e5d12f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:10.113176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247fed6-3c03-4718-9666-bd3c2e5d12f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:10.113515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247fed6-3c03-4718-9666-bd3c2e5d12f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:10.114763 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247fed6-3c03-4718-9666-bd3c2e5d12f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:22:10.115119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247fed6-3c03-4718-9666-bd3c2e5d12f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:10.116904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247fed6-3c03-4718-9666-bd3c2e5d12f3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:10.135711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1303/2607] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:22:10.137384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8e429-7f9a-42ed-a740-ccd33eb5dbb4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:10.147853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8e429-7f9a-42ed-a740-ccd33eb5dbb4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:10.152848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ceed1-d6ed-49bb-b288-e38b7aaf4d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:10.153519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ceed1-d6ed-49bb-b288-e38b7aaf4d71 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:10.153813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ceed1-d6ed-49bb-b288-e38b7aaf4d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:10.154090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ceed1-d6ed-49bb-b288-e38b7aaf4d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:10.154794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ceed1-d6ed-49bb-b288-e38b7aaf4d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:10.157319 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ceed1-d6ed-49bb-b288-e38b7aaf4d71 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:10.193089 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1305/2608] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:10 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:10.199596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247fed6-3c03-4718-9666-bd3c2e5d12f3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:10.211470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247fed6-3c03-4718-9666-bd3c2e5d12f3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:10.293216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf0041-eb5e-45b8-9b30-9252041f2025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:10.293904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf0041-eb5e-45b8-9b30-9252041f2025 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:10.294193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf0041-eb5e-45b8-9b30-9252041f2025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:10.295426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf0041-eb5e-45b8-9b30-9252041f2025 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:10.295426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf0041-eb5e-45b8-9b30-9252041f2025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:10.297326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf0041-eb5e-45b8-9b30-9252041f2025 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:10.329896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1304/2609] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:10 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:10.334754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ceed1-d6ed-49bb-b288-e38b7aaf4d71 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:10.351420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ceed1-d6ed-49bb-b288-e38b7aaf4d71 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:10.357127 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe79443-380a-46a1-87bf-97d5dbfa9340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:10.358014 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79443-380a-46a1-87bf-97d5dbfa9340 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:10.358423 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79443-380a-46a1-87bf-97d5dbfa9340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:10.358783 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79443-380a-46a1-87bf-97d5dbfa9340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:10.359708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe79443-380a-46a1-87bf-97d5dbfa9340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:10.361773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe79443-380a-46a1-87bf-97d5dbfa9340 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:10.373179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1306/2610] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:10.377904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf0041-eb5e-45b8-9b30-9252041f2025 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:10.388662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf0041-eb5e-45b8-9b30-9252041f2025 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:10.393744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688abf1a-498a-40ac-80ba-9226768f6f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:10.394409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688abf1a-498a-40ac-80ba-9226768f6f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:10.394706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688abf1a-498a-40ac-80ba-9226768f6f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:10.394988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688abf1a-498a-40ac-80ba-9226768f6f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:10.395754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688abf1a-498a-40ac-80ba-9226768f6f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:10.397353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688abf1a-498a-40ac-80ba-9226768f6f5a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:10.408018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1305/2611] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:10.412262 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe79443-380a-46a1-87bf-97d5dbfa9340 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:10.423586 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe79443-380a-46a1-87bf-97d5dbfa9340 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:10.428950 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef25f0-48cb-453c-a160-b3b776babf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:10.429876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef25f0-48cb-453c-a160-b3b776babf1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:10.430163 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef25f0-48cb-453c-a160-b3b776babf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:10.430458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef25f0-48cb-453c-a160-b3b776babf1a admin admin] PATH_INFO: `/v3/projects/1ff32118b2564a9fa237e8a3037044c2/users/7486a709a62a4f13b5b74ec503f3c952/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:10.448588 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef25f0-48cb-453c-a160-b3b776babf1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ff32118b2564a9fa237e8a3037044c2, user_id=7486a709a62a4f13b5b74ec503f3c952, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:10.451359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef25f0-48cb-453c-a160-b3b776babf1a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:10.466420 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1307/2612] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:10 2022] PUT /identity/v3/projects/1ff32118b2564a9fa237e8a3037044c2/users/7486a709a62a4f13b5b74ec503f3c952/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:10.485909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2fadd-c61e-4d8e-a983-ff611929a243 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:10.486427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2fadd-c61e-4d8e-a983-ff611929a243 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:10.486945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2fadd-c61e-4d8e-a983-ff611929a243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:10.525978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-68b2fadd-c61e-4d8e-a983-ff611929a243 None None] MFA Rules not processed for user `7486a709a62a4f13b5b74ec503f3c952`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:10.527227 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68b2fadd-c61e-4d8e-a983-ff611929a243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:10.598305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1306/2613] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:10.613936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef25f0-48cb-453c-a160-b3b776babf1a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:10.615927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d9ef25f0-48cb-453c-a160-b3b776babf1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:10.677301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef25f0-48cb-453c-a160-b3b776babf1a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:10.681206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635b94e-ae6b-49da-bcaf-4bf8f44de9c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:10.681859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635b94e-ae6b-49da-bcaf-4bf8f44de9c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:10.682150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635b94e-ae6b-49da-bcaf-4bf8f44de9c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:10.682661 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635b94e-ae6b-49da-bcaf-4bf8f44de9c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:10.692603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635b94e-ae6b-49da-bcaf-4bf8f44de9c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:10.694067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635b94e-ae6b-49da-bcaf-4bf8f44de9c2 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:10.703233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1308/2614] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:12.720040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2fadd-c61e-4d8e-a983-ff611929a243 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:12.732328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68b2fadd-c61e-4d8e-a983-ff611929a243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:12.823303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:12.839430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2fadd-c61e-4d8e-a983-ff611929a243 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:12.844163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:12.845653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:12.845653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:12.845653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] PATH_INFO: `/v3/users/0bf75458ad704a2bb0ca729ecdc3d498` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:12.846765 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf75458ad704a2bb0ca729ecdc3d498)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:12.848743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:12.900909 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf75458ad704a2bb0ca729ecdc3d498'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:12.908489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf75458ad704a2bb0ca729ecdc3d498'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:12.916317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf75458ad704a2bb0ca729ecdc3d498'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:12.928531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf75458ad704a2bb0ca729ecdc3d498'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:12.929293 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf75458ad704a2bb0ca729ecdc3d498'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:12.936743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1307/2615] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:12 2022] DELETE /identity/v3/users/0bf75458ad704a2bb0ca729ecdc3d498 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:12.942245 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6032-8615-4bd6-af83-29d030339881 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:12.942951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6032-8615-4bd6-af83-29d030339881 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:12.942951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6032-8615-4bd6-af83-29d030339881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:12.967416 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1bba6032-8615-4bd6-af83-29d030339881 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:12.968872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1bba6032-8615-4bd6-af83-29d030339881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:13.010446 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:13.048767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1309/2616] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:13.060313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:13.061783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:13.159065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45018f25-e942-48b1-92ef-54d73be6c9f6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:13.164196 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427a359-2821-4ff2-aeba-dfe5c4a34336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:13.164847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427a359-2821-4ff2-aeba-dfe5c4a34336 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:13.165165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427a359-2821-4ff2-aeba-dfe5c4a34336 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:13.165440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427a359-2821-4ff2-aeba-dfe5c4a34336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:13.175527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427a359-2821-4ff2-aeba-dfe5c4a34336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:13.178282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427a359-2821-4ff2-aeba-dfe5c4a34336 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:13.188657 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1308/2617] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:13.510400 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6032-8615-4bd6-af83-29d030339881 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:13.511919 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1bba6032-8615-4bd6-af83-29d030339881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:13.526817 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6032-8615-4bd6-af83-29d030339881 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:13.532588 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:13.534571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:13.534856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:13.535141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] PATH_INFO: `/v3/projects/504ab1a4133c4ca7b44b77300ecea2ec` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:13.536804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=504ab1a4133c4ca7b44b77300ecea2ec)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:13.538864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:13.588972 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504ab1a4133c4ca7b44b77300ecea2ec', 'request_id': 'req-44ff4d96-41b9-4774-8003-52274ae92c08'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.596385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504ab1a4133c4ca7b44b77300ecea2ec', 'request_id': 'req-44ff4d96-41b9-4774-8003-52274ae92c08'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.624518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504ab1a4133c4ca7b44b77300ecea2ec', 'request_id': 'req-44ff4d96-41b9-4774-8003-52274ae92c08'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.631073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] The token cache is being invalidate because project 504ab1a4133c4ca7b44b77300ecea2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:13.631399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.632052 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.633625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1310/2618] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:13 2022] DELETE /identity/v3/projects/504ab1a4133c4ca7b44b77300ecea2ec => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:13.637624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427a359-2821-4ff2-aeba-dfe5c4a34336 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:13.647502 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427a359-2821-4ff2-aeba-dfe5c4a34336 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:13.710191 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:13.721156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:13.722243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:13.722627 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:13.723087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] PATH_INFO: `/v3/users/c87704290bde424996fb99b8af7ae80f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:13.724835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87704290bde424996fb99b8af7ae80f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:13.726957 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:13.783285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87704290bde424996fb99b8af7ae80f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.793621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87704290bde424996fb99b8af7ae80f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.802420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87704290bde424996fb99b8af7ae80f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.820665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87704290bde424996fb99b8af7ae80f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.821412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f62a7ded-bb90-41c3-9009-b7de308f98ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87704290bde424996fb99b8af7ae80f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.828169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1309/2619] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:13 2022] DELETE /identity/v3/users/c87704290bde424996fb99b8af7ae80f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:13.840254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:13.842028 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:13.856168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff4d96-41b9-4774-8003-52274ae92c08 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:13.923531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:13.924456 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:13.924746 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:13.925026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] PATH_INFO: `/v3/users/7486a709a62a4f13b5b74ec503f3c952` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:13.926254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7486a709a62a4f13b5b74ec503f3c952)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:13.928239 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:13.991936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7486a709a62a4f13b5b74ec503f3c952'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:13.993107 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7486a709a62a4f13b5b74ec503f3c952'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.013363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7486a709a62a4f13b5b74ec503f3c952'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.016214 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:14.037468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7486a709a62a4f13b5b74ec503f3c952'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.048161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7486a709a62a4f13b5b74ec503f3c952'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.056067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1311/2620] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:13 2022] DELETE /identity/v3/users/7486a709a62a4f13b5b74ec503f3c952 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:14.061211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d018832-ae16-46e1-9617-f8ba68a28979 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:14.061211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d018832-ae16-46e1-9617-f8ba68a28979 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:14.061539 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d018832-ae16-46e1-9617-f8ba68a28979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:14.083722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8d018832-ae16-46e1-9617-f8ba68a28979 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:14.085368 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8d018832-ae16-46e1-9617-f8ba68a28979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:14.146542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:14.148584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:14.169195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1310/2621] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:14.183173 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d018832-ae16-46e1-9617-f8ba68a28979 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:14.184924 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8d018832-ae16-46e1-9617-f8ba68a28979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:14.239560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60788772-d8b2-439d-891f-7513527e6cc5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:14.245482 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:14.246872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:14.247407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:14.247772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] PATH_INFO: `/v3/projects/2d21b6f50e584af18aa71e957b001fcf` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:14.248127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d018832-ae16-46e1-9617-f8ba68a28979 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:14.249798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d21b6f50e584af18aa71e957b001fcf)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:14.254669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:14.254823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda0b1-81bc-4a63-9515-e730e69c7cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:14.254823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda0b1-81bc-4a63-9515-e730e69c7cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:14.254823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda0b1-81bc-4a63-9515-e730e69c7cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:14.255242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda0b1-81bc-4a63-9515-e730e69c7cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:14.267448 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bda0b1-81bc-4a63-9515-e730e69c7cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:14.269074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bda0b1-81bc-4a63-9515-e730e69c7cb4 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:14.281177 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1311/2622] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:14.306359 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d21b6f50e584af18aa71e957b001fcf', 'request_id': 'req-ec6ec661-4814-4377-a6af-802e08c554c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.315175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d21b6f50e584af18aa71e957b001fcf', 'request_id': 'req-ec6ec661-4814-4377-a6af-802e08c554c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.322794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d21b6f50e584af18aa71e957b001fcf', 'request_id': 'req-ec6ec661-4814-4377-a6af-802e08c554c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.339050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] The token cache is being invalidate because project 2d21b6f50e584af18aa71e957b001fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:14.339760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.340498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.342954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1312/2623] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:14 2022] DELETE /identity/v3/projects/2d21b6f50e584af18aa71e957b001fcf => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:14.346595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda0b1-81bc-4a63-9515-e730e69c7cb4 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:14.357663 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda0b1-81bc-4a63-9515-e730e69c7cb4 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:14.372258 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:14.436903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:14.438686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:14.438968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:14.439246 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] PATH_INFO: `/v3/users/a5d70ca2f58049f18eb7edd6d2ee124b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:14.440856 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d70ca2f58049f18eb7edd6d2ee124b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:14.442833 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:14.513927 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d70ca2f58049f18eb7edd6d2ee124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.526116 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d70ca2f58049f18eb7edd6d2ee124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.536055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d70ca2f58049f18eb7edd6d2ee124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.550550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d70ca2f58049f18eb7edd6d2ee124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.551299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34774aee-b738-4164-9387-61d8a93b75c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d70ca2f58049f18eb7edd6d2ee124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.559690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1312/2624] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:14 2022] DELETE /identity/v3/users/a5d70ca2f58049f18eb7edd6d2ee124b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:14.610454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:14.611975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:14.626694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ec661-4814-4377-a6af-802e08c554c7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:14.714334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:14.715289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:14.715859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:14.715859 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] PATH_INFO: `/v3/projects/1ff32118b2564a9fa237e8a3037044c2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:14.717395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ff32118b2564a9fa237e8a3037044c2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:14.719508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:14.726540 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:14.781141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ff32118b2564a9fa237e8a3037044c2', 'request_id': 'req-571ba4d3-77ed-4507-8d9c-e392e4b4c419'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.793209 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ff32118b2564a9fa237e8a3037044c2', 'request_id': 'req-571ba4d3-77ed-4507-8d9c-e392e4b4c419'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.802530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ff32118b2564a9fa237e8a3037044c2', 'request_id': 'req-571ba4d3-77ed-4507-8d9c-e392e4b4c419'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.817933 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] The token cache is being invalidate because project 1ff32118b2564a9fa237e8a3037044c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:14.818231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.818875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:14.820523 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1313/2625] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:14 2022] DELETE /identity/v3/projects/1ff32118b2564a9fa237e8a3037044c2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:14.825653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62a58d-5595-41a1-922c-9bb031d30599 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:14.826213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62a58d-5595-41a1-922c-9bb031d30599 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:14.826370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62a58d-5595-41a1-922c-9bb031d30599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:14.845187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dd62a58d-5595-41a1-922c-9bb031d30599 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:14.846434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dd62a58d-5595-41a1-922c-9bb031d30599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:14.907799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:14.909757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:14.925771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1313/2626] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:14.934211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c6ce6-b9c4-4df2-a822-779ce4f0cfb8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:14.934449 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c6ce6-b9c4-4df2-a822-779ce4f0cfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:14.934738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c6ce6-b9c4-4df2-a822-779ce4f0cfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:14.954229 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6e1c6ce6-b9c4-4df2-a822-779ce4f0cfb8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:14.955599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6e1c6ce6-b9c4-4df2-a822-779ce4f0cfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:14.963951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1314/2627] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:14.968541 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c6ce6-b9c4-4df2-a822-779ce4f0cfb8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:14.978897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c6ce6-b9c4-4df2-a822-779ce4f0cfb8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:14.985469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77cbc4-ca97-474a-b412-6ebb9d0f7a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:14.986169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77cbc4-ca97-474a-b412-6ebb9d0f7a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:14.986467 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77cbc4-ca97-474a-b412-6ebb9d0f7a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:14.986748 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77cbc4-ca97-474a-b412-6ebb9d0f7a7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:14.987576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77cbc4-ca97-474a-b412-6ebb9d0f7a7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:22:14.987887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77cbc4-ca97-474a-b412-6ebb9d0f7a7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:14.989778 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77cbc4-ca97-474a-b412-6ebb9d0f7a7b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:14.999536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ba4d3-77ed-4507-8d9c-e392e4b4c419 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.003090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1315/2628] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 22 11:22:15.006695 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77cbc4-ca97-474a-b412-6ebb9d0f7a7b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.007959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.008845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.009134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.009415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] PATH_INFO: `/v3/projects/9b730b8aa59c4e148409aaa63daa9292` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.010633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b730b8aa59c4e148409aaa63daa9292)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.012640 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.017099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77cbc4-ca97-474a-b412-6ebb9d0f7a7b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.022150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ede78-4011-43b2-8171-01b8467550c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.023464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ede78-4011-43b2-8171-01b8467550c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.023464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ede78-4011-43b2-8171-01b8467550c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.023464 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ede78-4011-43b2-8171-01b8467550c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.024161 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ede78-4011-43b2-8171-01b8467550c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.026204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ede78-4011-43b2-8171-01b8467550c6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.042037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:15.064355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1316/2629] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:15 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:15.068901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ede78-4011-43b2-8171-01b8467550c6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.076659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b730b8aa59c4e148409aaa63daa9292', 'request_id': 'req-87f6311c-538e-4b16-ba67-62554b64ef09'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:15.083074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b730b8aa59c4e148409aaa63daa9292', 'request_id': 'req-87f6311c-538e-4b16-ba67-62554b64ef09'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:15.089352 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b730b8aa59c4e148409aaa63daa9292', 'request_id': 'req-87f6311c-538e-4b16-ba67-62554b64ef09'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:15.106342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] The token cache is being invalidate because project 9b730b8aa59c4e148409aaa63daa9292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:15.106666 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:15.107614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-87f6311c-538e-4b16-ba67-62554b64ef09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:15.110234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1314/2630] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:14 2022] DELETE /identity/v3/projects/9b730b8aa59c4e148409aaa63daa9292 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:15.115244 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30414d7e-1cce-4678-a043-207e4b45e54d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.115566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30414d7e-1cce-4678-a043-207e4b45e54d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.115848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30414d7e-1cce-4678-a043-207e4b45e54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.140573 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-30414d7e-1cce-4678-a043-207e4b45e54d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:15.141771 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ede78-4011-43b2-8171-01b8467550c6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.142160 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-30414d7e-1cce-4678-a043-207e4b45e54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:15.146636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afef0f-699c-4204-b852-69f33e62a124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.147356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afef0f-699c-4204-b852-69f33e62a124 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.147631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afef0f-699c-4204-b852-69f33e62a124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.147920 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afef0f-699c-4204-b852-69f33e62a124 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.148612 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02afef0f-699c-4204-b852-69f33e62a124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.150576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02afef0f-699c-4204-b852-69f33e62a124 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.151653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1315/2631] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:15.161241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae702e-8696-4565-8f8a-303cd166e4e8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.161928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae702e-8696-4565-8f8a-303cd166e4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.162330 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae702e-8696-4565-8f8a-303cd166e4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.177753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1317/2632] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:15 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:15.182087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afef0f-699c-4204-b852-69f33e62a124 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.184581 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-02afef0f-699c-4204-b852-69f33e62a124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:15.185448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-19ae702e-8696-4565-8f8a-303cd166e4e8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:15.186645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-19ae702e-8696-4565-8f8a-303cd166e4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:15.195519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1316/2633] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:15.199509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afef0f-699c-4204-b852-69f33e62a124 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.200475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae702e-8696-4565-8f8a-303cd166e4e8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.204426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27975495-9951-44bd-96db-3d6e9e458f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.205111 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27975495-9951-44bd-96db-3d6e9e458f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.205454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27975495-9951-44bd-96db-3d6e9e458f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.205792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27975495-9951-44bd-96db-3d6e9e458f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.206736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27975495-9951-44bd-96db-3d6e9e458f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.208434 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27975495-9951-44bd-96db-3d6e9e458f7f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.211652 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae702e-8696-4565-8f8a-303cd166e4e8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.216581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da84d5-578a-46dc-9344-403feb0fbe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.217227 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da84d5-578a-46dc-9344-403feb0fbe7b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.217513 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da84d5-578a-46dc-9344-403feb0fbe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.217791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da84d5-578a-46dc-9344-403feb0fbe7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.218503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da84d5-578a-46dc-9344-403feb0fbe7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:22:15.219349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da84d5-578a-46dc-9344-403feb0fbe7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.222576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1318/2634] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:15.222713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da84d5-578a-46dc-9344-403feb0fbe7b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.223413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27975495-9951-44bd-96db-3d6e9e458f7f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.229801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:15.233383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27975495-9951-44bd-96db-3d6e9e458f7f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.238896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1317/2635] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:22:15.244043 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da84d5-578a-46dc-9344-403feb0fbe7b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.244621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ebc50-d4e2-42a9-8377-291c9c3e6b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.245405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ebc50-d4e2-42a9-8377-291c9c3e6b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.245697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ebc50-d4e2-42a9-8377-291c9c3e6b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.246096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ebc50-d4e2-42a9-8377-291c9c3e6b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.246839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ebc50-d4e2-42a9-8377-291c9c3e6b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.248529 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ebc50-d4e2-42a9-8377-291c9c3e6b4b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.255923 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da84d5-578a-46dc-9344-403feb0fbe7b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.260621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599718c7-8960-400b-8617-5c96f2b5ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.261216 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1319/2636] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:15.261363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599718c7-8960-400b-8617-5c96f2b5ea76 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.261560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599718c7-8960-400b-8617-5c96f2b5ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.261847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599718c7-8960-400b-8617-5c96f2b5ea76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.262714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599718c7-8960-400b-8617-5c96f2b5ea76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.264559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599718c7-8960-400b-8617-5c96f2b5ea76 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.265318 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ebc50-d4e2-42a9-8377-291c9c3e6b4b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.277094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ebc50-d4e2-42a9-8377-291c9c3e6b4b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.283017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fa67e-4e1b-4f71-9a8b-29741f172e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.283944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fa67e-4e1b-4f71-9a8b-29741f172e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.284232 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fa67e-4e1b-4f71-9a8b-29741f172e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.284564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fa67e-4e1b-4f71-9a8b-29741f172e8e admin admin] PATH_INFO: `/v3/projects/bffa24b2792d47ff8bce2d2da219be64/users/16e3aabf2a2d4e91a9b5dc61543f4024/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.297402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1318/2637] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:15 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:15.301919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fa67e-4e1b-4f71-9a8b-29741f172e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bffa24b2792d47ff8bce2d2da219be64, user_id=16e3aabf2a2d4e91a9b5dc61543f4024, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.303355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599718c7-8960-400b-8617-5c96f2b5ea76 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.304895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fa67e-4e1b-4f71-9a8b-29741f172e8e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.323046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1320/2638] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:15 2022] PUT /identity/v3/projects/bffa24b2792d47ff8bce2d2da219be64/users/16e3aabf2a2d4e91a9b5dc61543f4024/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:15.335884 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345989e3-ea8c-498b-bc3f-e818554a2905 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.336479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345989e3-ea8c-498b-bc3f-e818554a2905 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.336774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345989e3-ea8c-498b-bc3f-e818554a2905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.375443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-345989e3-ea8c-498b-bc3f-e818554a2905 None None] MFA Rules not processed for user `16e3aabf2a2d4e91a9b5dc61543f4024`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:15.376740 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-345989e3-ea8c-498b-bc3f-e818554a2905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:15.392065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599718c7-8960-400b-8617-5c96f2b5ea76 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.454390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1321/2639] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:15.470104 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345989e3-ea8c-498b-bc3f-e818554a2905 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.471632 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-345989e3-ea8c-498b-bc3f-e818554a2905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:15.473384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385f94d-8e13-4d61-8e79-52a40941c9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.474183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385f94d-8e13-4d61-8e79-52a40941c9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.474509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385f94d-8e13-4d61-8e79-52a40941c9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.474765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385f94d-8e13-4d61-8e79-52a40941c9f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.475499 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385f94d-8e13-4d61-8e79-52a40941c9f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.477662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385f94d-8e13-4d61-8e79-52a40941c9f9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.514248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1319/2640] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:15 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:15.514458 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:15.517134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385f94d-8e13-4d61-8e79-52a40941c9f9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.528998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385f94d-8e13-4d61-8e79-52a40941c9f9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.534437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a564a16-9bc9-4bbd-8fb4-ce52e9013e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.535103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a564a16-9bc9-4bbd-8fb4-ce52e9013e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.535433 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a564a16-9bc9-4bbd-8fb4-ce52e9013e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.535718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a564a16-9bc9-4bbd-8fb4-ce52e9013e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.536450 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a564a16-9bc9-4bbd-8fb4-ce52e9013e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.538045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a564a16-9bc9-4bbd-8fb4-ce52e9013e9d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.547804 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345989e3-ea8c-498b-bc3f-e818554a2905 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.548722 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1320/2641] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:15.552163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d211fe8-2639-48c4-a7c1-d75469b340a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.552826 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d211fe8-2639-48c4-a7c1-d75469b340a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.553110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d211fe8-2639-48c4-a7c1-d75469b340a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.553388 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d211fe8-2639-48c4-a7c1-d75469b340a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.553565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a564a16-9bc9-4bbd-8fb4-ce52e9013e9d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.562639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d211fe8-2639-48c4-a7c1-d75469b340a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.564112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d211fe8-2639-48c4-a7c1-d75469b340a6 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.564754 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a564a16-9bc9-4bbd-8fb4-ce52e9013e9d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.569629 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e81e-adcc-404d-b1b4-941053518acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.570348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e81e-adcc-404d-b1b4-941053518acb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.570643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e81e-adcc-404d-b1b4-941053518acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.570934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e81e-adcc-404d-b1b4-941053518acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.571703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82e81e-adcc-404d-b1b4-941053518acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.573295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82e81e-adcc-404d-b1b4-941053518acb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.574492 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1322/2642] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:15.585583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1321/2643] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:15.603354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d211fe8-2639-48c4-a7c1-d75469b340a6 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.621520 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d211fe8-2639-48c4-a7c1-d75469b340a6 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.627934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28e926-d997-483d-bf69-25aea1e0f49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.628835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28e926-d997-483d-bf69-25aea1e0f49e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.629124 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28e926-d997-483d-bf69-25aea1e0f49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.629404 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28e926-d997-483d-bf69-25aea1e0f49e admin admin] PATH_INFO: `/v3/projects/17c28a2aaa4b4d4bada90cea2b8ccff5/users/2e46f6bf93bf4e839a0dfebd5409f32e/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.646546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28e926-d997-483d-bf69-25aea1e0f49e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c28a2aaa4b4d4bada90cea2b8ccff5, user_id=2e46f6bf93bf4e839a0dfebd5409f32e, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.649247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28e926-d997-483d-bf69-25aea1e0f49e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.668500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1323/2644] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:15 2022] PUT /identity/v3/projects/17c28a2aaa4b4d4bada90cea2b8ccff5/users/2e46f6bf93bf4e839a0dfebd5409f32e/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:15.674139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38d23b-f566-4799-8aba-61ccd95e6b43 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.674720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38d23b-f566-4799-8aba-61ccd95e6b43 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.675006 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38d23b-f566-4799-8aba-61ccd95e6b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.693681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2b38d23b-f566-4799-8aba-61ccd95e6b43 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:15.694891 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2b38d23b-f566-4799-8aba-61ccd95e6b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:15.759036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:15.780473 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1322/2645] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:15.791870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28e926-d997-483d-bf69-25aea1e0f49e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:15.794210 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1d28e926-d997-483d-bf69-25aea1e0f49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:15.894696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28e926-d997-483d-bf69-25aea1e0f49e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:15.899644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a7aad-d023-43a6-89e7-c2a47fcbc320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:15.900600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a7aad-d023-43a6-89e7-c2a47fcbc320 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:15.900964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a7aad-d023-43a6-89e7-c2a47fcbc320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:15.901324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a7aad-d023-43a6-89e7-c2a47fcbc320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:15.915840 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a7aad-d023-43a6-89e7-c2a47fcbc320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:15.918019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a7aad-d023-43a6-89e7-c2a47fcbc320 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:15.928348 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1324/2646] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:16.620695 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4c3e2-5712-4e89-80fb-fd8089556e69 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:16.621205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4c3e2-5712-4e89-80fb-fd8089556e69 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:16.621331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4c3e2-5712-4e89-80fb-fd8089556e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:16.660892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-90c4c3e2-5712-4e89-80fb-fd8089556e69 None None] MFA Rules not processed for user `2e46f6bf93bf4e839a0dfebd5409f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:16.662133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-90c4c3e2-5712-4e89-80fb-fd8089556e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:16.741848 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1323/2647] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:16.759355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a7aad-d023-43a6-89e7-c2a47fcbc320 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:16.834486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a7aad-d023-43a6-89e7-c2a47fcbc320 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:16.838566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dbf90-d3ae-4191-9b10-0882a8e69c25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:16.839230 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dbf90-d3ae-4191-9b10-0882a8e69c25 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:16.839575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dbf90-d3ae-4191-9b10-0882a8e69c25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:16.839863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dbf90-d3ae-4191-9b10-0882a8e69c25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:16.850542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dbf90-d3ae-4191-9b10-0882a8e69c25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:16.852023 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dbf90-d3ae-4191-9b10-0882a8e69c25 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:16.861435 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1325/2648] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:18.219232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4c3e2-5712-4e89-80fb-fd8089556e69 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:18.230944 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4c3e2-5712-4e89-80fb-fd8089556e69 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:18.236664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:18.237542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:18.237856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:18.238140 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] PATH_INFO: `/v3/users/16e3aabf2a2d4e91a9b5dc61543f4024` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:18.239665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e3aabf2a2d4e91a9b5dc61543f4024)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:18.241739 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:18.278904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383d1b4-2405-4dd9-b429-c351e1c816b1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:18.279816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383d1b4-2405-4dd9-b429-c351e1c816b1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:18.280011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383d1b4-2405-4dd9-b429-c351e1c816b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:18.296636 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e3aabf2a2d4e91a9b5dc61543f4024'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:18.297328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e3aabf2a2d4e91a9b5dc61543f4024'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:18.300028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0383d1b4-2405-4dd9-b429-c351e1c816b1 None None] MFA Rules not processed for user `2e46f6bf93bf4e839a0dfebd5409f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:18.301690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0383d1b4-2405-4dd9-b429-c351e1c816b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:18.304757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e3aabf2a2d4e91a9b5dc61543f4024'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:18.311069 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1326/2649] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:18.322146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e3aabf2a2d4e91a9b5dc61543f4024'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:18.330214 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e3aabf2a2d4e91a9b5dc61543f4024'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:18.337988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1324/2650] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:18 2022] DELETE /identity/v3/users/16e3aabf2a2d4e91a9b5dc61543f4024 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:18.342889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc74634-8151-484e-87af-67597bb96464 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:18.343370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc74634-8151-484e-87af-67597bb96464 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:18.344297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc74634-8151-484e-87af-67597bb96464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:18.364194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4fc74634-8151-484e-87af-67597bb96464 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:18.365669 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4fc74634-8151-484e-87af-67597bb96464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:18.412796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:18.441470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1327/2651] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:18.448323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:18.449937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:18.517318 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f3edf-e03a-47f0-8a7b-ea857677500d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:18.521064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d304a0-2000-4ab9-9c48-204398f3d835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:18.521718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d304a0-2000-4ab9-9c48-204398f3d835 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:18.522002 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d304a0-2000-4ab9-9c48-204398f3d835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:18.522281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d304a0-2000-4ab9-9c48-204398f3d835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:18.530559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:18.538444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d304a0-2000-4ab9-9c48-204398f3d835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:18.542036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d304a0-2000-4ab9-9c48-204398f3d835 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:18.562862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1325/2652] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:18.895022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc74634-8151-484e-87af-67597bb96464 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:18.896870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4fc74634-8151-484e-87af-67597bb96464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:18.932753 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc74634-8151-484e-87af-67597bb96464 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:18.957118 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:18.958122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:18.958962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:18.959654 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] PATH_INFO: `/v3/projects/bffa24b2792d47ff8bce2d2da219be64` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:18.963873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bffa24b2792d47ff8bce2d2da219be64)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:18.966242 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:19.036567 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bffa24b2792d47ff8bce2d2da219be64', 'request_id': 'req-81d7ab45-ff96-4b01-ac5e-ede523d847d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:19.043665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bffa24b2792d47ff8bce2d2da219be64', 'request_id': 'req-81d7ab45-ff96-4b01-ac5e-ede523d847d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:19.050715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bffa24b2792d47ff8bce2d2da219be64', 'request_id': 'req-81d7ab45-ff96-4b01-ac5e-ede523d847d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:19.068369 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] The token cache is being invalidate because project bffa24b2792d47ff8bce2d2da219be64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:19.068670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:19.069468 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81d7ab45-ff96-4b01-ac5e-ede523d847d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:19.071035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1328/2653] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:18 2022] DELETE /identity/v3/projects/bffa24b2792d47ff8bce2d2da219be64 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:19.078885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6cc76-d079-4672-bbaa-707e248143f7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.079431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6cc76-d079-4672-bbaa-707e248143f7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.079943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6cc76-d079-4672-bbaa-707e248143f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.101436 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b3f6cc76-d079-4672-bbaa-707e248143f7 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:19.103416 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b3f6cc76-d079-4672-bbaa-707e248143f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:19.185612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1326/2654] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:19.197787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafa327-f10c-437c-8eec-66862e37186b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.197787 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafa327-f10c-437c-8eec-66862e37186b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.198413 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafa327-f10c-437c-8eec-66862e37186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.219451 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8aafa327-f10c-437c-8eec-66862e37186b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:19.221053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8aafa327-f10c-437c-8eec-66862e37186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:19.235220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1329/2655] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:22:19.240849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6cc76-d079-4672-bbaa-707e248143f7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:19.252114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6cc76-d079-4672-bbaa-707e248143f7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:19.257193 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6318c0-9fdd-4cce-a4d5-df0ee1090b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:19.257868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6318c0-9fdd-4cce-a4d5-df0ee1090b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.258177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6318c0-9fdd-4cce-a4d5-df0ee1090b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.258454 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6318c0-9fdd-4cce-a4d5-df0ee1090b2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.259484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6318c0-9fdd-4cce-a4d5-df0ee1090b2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:22:19.259686 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6318c0-9fdd-4cce-a4d5-df0ee1090b2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:19.261699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6318c0-9fdd-4cce-a4d5-df0ee1090b2f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:19.279403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1327/2656] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:22:19.282809 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafa327-f10c-437c-8eec-66862e37186b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:19.294499 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafa327-f10c-437c-8eec-66862e37186b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:19.301276 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee47e0-b31e-49fe-adda-b7050a928a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:19.302066 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee47e0-b31e-49fe-adda-b7050a928a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.302472 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee47e0-b31e-49fe-adda-b7050a928a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.302845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee47e0-b31e-49fe-adda-b7050a928a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.303743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee47e0-b31e-49fe-adda-b7050a928a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:19.305735 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee47e0-b31e-49fe-adda-b7050a928a08 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:19.355325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1330/2657] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:19 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:19.404304 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6318c0-9fdd-4cce-a4d5-df0ee1090b2f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:19.415171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6318c0-9fdd-4cce-a4d5-df0ee1090b2f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:19.484025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa24a2-b0fc-404b-ae3a-ce82f4bd47d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:19.485334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa24a2-b0fc-404b-ae3a-ce82f4bd47d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.485559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa24a2-b0fc-404b-ae3a-ce82f4bd47d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.485800 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa24a2-b0fc-404b-ae3a-ce82f4bd47d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.486514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa24a2-b0fc-404b-ae3a-ce82f4bd47d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:19.488521 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa24a2-b0fc-404b-ae3a-ce82f4bd47d8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:19.509350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1328/2658] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:19 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:19.513692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee47e0-b31e-49fe-adda-b7050a928a08 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:19.519472 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa24a2-b0fc-404b-ae3a-ce82f4bd47d8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:19.521844 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0afa24a2-b0fc-404b-ae3a-ce82f4bd47d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:19.524652 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee47e0-b31e-49fe-adda-b7050a928a08 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:19.529315 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bce43-9992-460f-8348-9f790fc63aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:19.529978 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bce43-9992-460f-8348-9f790fc63aae admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.530262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bce43-9992-460f-8348-9f790fc63aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.530563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bce43-9992-460f-8348-9f790fc63aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.531976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2bce43-9992-460f-8348-9f790fc63aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:19.533583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2bce43-9992-460f-8348-9f790fc63aae admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:19.546217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1331/2659] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:19.550874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bce43-9992-460f-8348-9f790fc63aae admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:19.561193 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bce43-9992-460f-8348-9f790fc63aae admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:19.565930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf65117-0602-4495-a5be-82dbc2e70fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:19.566590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf65117-0602-4495-a5be-82dbc2e70fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.566870 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf65117-0602-4495-a5be-82dbc2e70fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.567147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf65117-0602-4495-a5be-82dbc2e70fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.567902 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf65117-0602-4495-a5be-82dbc2e70fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:19.569504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf65117-0602-4495-a5be-82dbc2e70fc0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:19.579755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1332/2660] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:19.583969 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf65117-0602-4495-a5be-82dbc2e70fc0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:19.596265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf65117-0602-4495-a5be-82dbc2e70fc0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:19.599164 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d47fff-b688-47c4-aa4e-0199b38f343a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:19.600094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d47fff-b688-47c4-aa4e-0199b38f343a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.600580 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d47fff-b688-47c4-aa4e-0199b38f343a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.600717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d47fff-b688-47c4-aa4e-0199b38f343a admin admin] PATH_INFO: `/v3/projects/ee8f2050ff1d48c7871edba63439e007/users/5b6bfa46cc3849d2a9b11fd41912722e/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.614997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:19.617993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d47fff-b688-47c4-aa4e-0199b38f343a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8f2050ff1d48c7871edba63439e007, user_id=5b6bfa46cc3849d2a9b11fd41912722e, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:19.620692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d47fff-b688-47c4-aa4e-0199b38f343a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:19.628109 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa24a2-b0fc-404b-ae3a-ce82f4bd47d8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:19.636150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1333/2661] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:19 2022] PUT /identity/v3/projects/ee8f2050ff1d48c7871edba63439e007/users/5b6bfa46cc3849d2a9b11fd41912722e/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:22:19.646205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258cab-fd7f-402e-9665-5847387586b6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.646503 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258cab-fd7f-402e-9665-5847387586b6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.646872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0258cab-fd7f-402e-9665-5847387586b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.674515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e0258cab-fd7f-402e-9665-5847387586b6 None None] MFA Rules not processed for user `5b6bfa46cc3849d2a9b11fd41912722e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:19.675796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e0258cab-fd7f-402e-9665-5847387586b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:19.692419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:19.693039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.693322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.693633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.695444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:19.706877 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:19.708551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:19.719758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1329/2662] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:19.723688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:19.754273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1334/2663] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:19.766124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:19.767977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:19.878275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd02eb1-5f67-4e3e-b38b-7aa74640c246 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:19.883370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb2972-5c5c-4a74-944d-e722063c25cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:19.884164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb2972-5c5c-4a74-944d-e722063c25cb service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:19.884457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb2972-5c5c-4a74-944d-e722063c25cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:19.884741 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb2972-5c5c-4a74-944d-e722063c25cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:19.895623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb2972-5c5c-4a74-944d-e722063c25cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:19.897141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb2972-5c5c-4a74-944d-e722063c25cb service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:19.908281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1330/2664] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:21.332960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3da74-0f65-4536-9476-48040ca28fe4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:21.333518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3da74-0f65-4536-9476-48040ca28fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:21.333891 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3da74-0f65-4536-9476-48040ca28fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:21.354362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e4b3da74-0f65-4536-9476-48040ca28fe4 None None] MFA Rules not processed for user `5b6bfa46cc3849d2a9b11fd41912722e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:21.355675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e4b3da74-0f65-4536-9476-48040ca28fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:21.363046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1335/2665] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:29.902728 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=69 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:31.348422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=11 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:35.229040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d270de-09d1-4940-acfd-d9792414399a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:35.229602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d270de-09d1-4940-acfd-d9792414399a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:35.229602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d270de-09d1-4940-acfd-d9792414399a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:35.250875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-17d270de-09d1-4940-acfd-d9792414399a None None] MFA Rules not processed for user `2e46f6bf93bf4e839a0dfebd5409f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:35.252067 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-17d270de-09d1-4940-acfd-d9792414399a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:35.315821 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1331/2666] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:39.662732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69b108-e23e-4cae-8b9e-65b82683e98b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:39.663324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69b108-e23e-4cae-8b9e-65b82683e98b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:39.663721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69b108-e23e-4cae-8b9e-65b82683e98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:39.689987 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2c69b108-e23e-4cae-8b9e-65b82683e98b None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:39.691301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c69b108-e23e-4cae-8b9e-65b82683e98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:39.757397 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1336/2667] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:22:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:39.782615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d270de-09d1-4940-acfd-d9792414399a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:39.785161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-17d270de-09d1-4940-acfd-d9792414399a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:39.870317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d270de-09d1-4940-acfd-d9792414399a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:39.876384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac506b5-1b63-4866-8b83-7f553dcef500 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:39.877328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac506b5-1b63-4866-8b83-7f553dcef500 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:39.877328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac506b5-1b63-4866-8b83-7f553dcef500 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:39.877603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac506b5-1b63-4866-8b83-7f553dcef500 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:39.886564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac506b5-1b63-4866-8b83-7f553dcef500 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:39.888233 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac506b5-1b63-4866-8b83-7f553dcef500 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:39.897368 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1332/2668] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:22:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:22:40.537253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543a7ad-12a7-48e4-bf29-d5d523bec026 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:40.537553 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543a7ad-12a7-48e4-bf29-d5d523bec026 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:40.537846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543a7ad-12a7-48e4-bf29-d5d523bec026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:40.556150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8543a7ad-12a7-48e4-bf29-d5d523bec026 None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:40.557446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8543a7ad-12a7-48e4-bf29-d5d523bec026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:40.566405 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1337/2669] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 22 11:22:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:40.581642 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac506b5-1b63-4866-8b83-7f553dcef500 service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:40.598566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac506b5-1b63-4866-8b83-7f553dcef500 service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:40.606070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf7871-8ec7-47af-81e0-be25056ff124 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:40.607202 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf7871-8ec7-47af-81e0-be25056ff124 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:40.607743 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf7871-8ec7-47af-81e0-be25056ff124 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:40.608256 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf7871-8ec7-47af-81e0-be25056ff124 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:40.622819 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf7871-8ec7-47af-81e0-be25056ff124 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:40.623432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf7871-8ec7-47af-81e0-be25056ff124 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:40.634082 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1333/2670] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:22:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 22 11:22:40.907287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa06e2-c269-41f0-9bb8-dc2b14432c96 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:40.907755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa06e2-c269-41f0-9bb8-dc2b14432c96 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:40.907986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa06e2-c269-41f0-9bb8-dc2b14432c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:40.927693 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-70fa06e2-c269-41f0-9bb8-dc2b14432c96 None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:40.928910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-70fa06e2-c269-41f0-9bb8-dc2b14432c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:40.936574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1338/2671] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:22:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:40.950643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf7871-8ec7-47af-81e0-be25056ff124 service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:40.971511 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf7871-8ec7-47af-81e0-be25056ff124 service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:40.976802 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb1bed-463e-402e-8ce7-a23fe32a6f2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:40.977459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb1bed-463e-402e-8ce7-a23fe32a6f2d service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:40.977751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb1bed-463e-402e-8ce7-a23fe32a6f2d service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:40.978034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb1bed-463e-402e-8ce7-a23fe32a6f2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:40.987703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb1bed-463e-402e-8ce7-a23fe32a6f2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:40.989556 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb1bed-463e-402e-8ce7-a23fe32a6f2d service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:41.001456 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1334/2672] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:22:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 22 11:22:41.374380 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412ffda-c7c4-4b88-bc50-9768d08d5fca None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:41.374679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412ffda-c7c4-4b88-bc50-9768d08d5fca None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:41.374967 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412ffda-c7c4-4b88-bc50-9768d08d5fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:41.394813 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3412ffda-c7c4-4b88-bc50-9768d08d5fca None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:41.396170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3412ffda-c7c4-4b88-bc50-9768d08d5fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:41.403491 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1339/2673] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 22 11:22:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:41.416794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb1bed-463e-402e-8ce7-a23fe32a6f2d service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:41.430020 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb1bed-463e-402e-8ce7-a23fe32a6f2d service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:41.436397 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b37c9d-933e-4688-9bb1-2209d918af7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:41.437677 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b37c9d-933e-4688-9bb1-2209d918af7e service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:41.438080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b37c9d-933e-4688-9bb1-2209d918af7e service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:41.438492 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b37c9d-933e-4688-9bb1-2209d918af7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:41.450377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b37c9d-933e-4688-9bb1-2209d918af7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:41.452146 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b37c9d-933e-4688-9bb1-2209d918af7e service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:41.461309 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1335/2674] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:22:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:22:43.715931 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412ffda-c7c4-4b88-bc50-9768d08d5fca None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:43.791929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412ffda-c7c4-4b88-bc50-9768d08d5fca None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:43.797280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:43.798255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:43.798784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:43.799045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] PATH_INFO: `/v3/users/5b6bfa46cc3849d2a9b11fd41912722e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:43.800365 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b6bfa46cc3849d2a9b11fd41912722e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:43.802607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:43.860986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b6bfa46cc3849d2a9b11fd41912722e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:43.869661 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b6bfa46cc3849d2a9b11fd41912722e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:43.877899 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b6bfa46cc3849d2a9b11fd41912722e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:43.890212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b6bfa46cc3849d2a9b11fd41912722e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:43.890936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b6bfa46cc3849d2a9b11fd41912722e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:43.896964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1340/2675] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:43 2022] DELETE /identity/v3/users/5b6bfa46cc3849d2a9b11fd41912722e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:43.901488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8988f-68f9-49d0-a8e4-6645d697e0c5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:43.902129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8988f-68f9-49d0-a8e4-6645d697e0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:43.902129 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8988f-68f9-49d0-a8e4-6645d697e0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:43.920297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-18c8988f-68f9-49d0-a8e4-6645d697e0c5 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:43.922065 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-18c8988f-68f9-49d0-a8e4-6645d697e0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:43.975269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:43.988385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1336/2676] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:44.000998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:44.002904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:44.010461 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:44.073752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab9389-a2d9-49b7-a982-cecac76e821b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:44.077843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cd3b7-6ec5-455c-b593-d436aad81043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:44.078500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cd3b7-6ec5-455c-b593-d436aad81043 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:44.079019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cd3b7-6ec5-455c-b593-d436aad81043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:44.079353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cd3b7-6ec5-455c-b593-d436aad81043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:44.091287 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cd3b7-6ec5-455c-b593-d436aad81043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:44.093802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cd3b7-6ec5-455c-b593-d436aad81043 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:44.106285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1341/2677] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:44.497547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8988f-68f9-49d0-a8e4-6645d697e0c5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:44.500864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-18c8988f-68f9-49d0-a8e4-6645d697e0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:44.515818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8988f-68f9-49d0-a8e4-6645d697e0c5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:44.522882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:44.524086 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:44.524768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:44.525223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] PATH_INFO: `/v3/projects/ee8f2050ff1d48c7871edba63439e007` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:44.526669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8f2050ff1d48c7871edba63439e007)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:44.529443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:44.618011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8f2050ff1d48c7871edba63439e007', 'request_id': 'req-225ac976-ef21-49bc-87c0-f321a9ec556d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:44.625576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8f2050ff1d48c7871edba63439e007', 'request_id': 'req-225ac976-ef21-49bc-87c0-f321a9ec556d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:44.632790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8f2050ff1d48c7871edba63439e007', 'request_id': 'req-225ac976-ef21-49bc-87c0-f321a9ec556d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:44.652409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] The token cache is being invalidate because project ee8f2050ff1d48c7871edba63439e007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:44.652963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:44.653828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-225ac976-ef21-49bc-87c0-f321a9ec556d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:44.656602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1337/2678] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:44 2022] DELETE /identity/v3/projects/ee8f2050ff1d48c7871edba63439e007 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:44.663439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32090383-2d7e-4d49-b872-73ef6ce987b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:44.664271 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32090383-2d7e-4d49-b872-73ef6ce987b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:44.664831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32090383-2d7e-4d49-b872-73ef6ce987b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:44.698828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-32090383-2d7e-4d49-b872-73ef6ce987b9 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:44.700145 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-32090383-2d7e-4d49-b872-73ef6ce987b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:44.775292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1342/2679] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:44.783462 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a463bc1-2280-482c-b20f-6431e8f16d11 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:44.783762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a463bc1-2280-482c-b20f-6431e8f16d11 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:44.784050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a463bc1-2280-482c-b20f-6431e8f16d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:44.805405 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3a463bc1-2280-482c-b20f-6431e8f16d11 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:44.806643 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3a463bc1-2280-482c-b20f-6431e8f16d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:44.815108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1338/2680] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:44.820975 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32090383-2d7e-4d49-b872-73ef6ce987b9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:44.837151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32090383-2d7e-4d49-b872-73ef6ce987b9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:44.844809 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9004-d15b-4326-ac4a-ae77438d1bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:44.845810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9004-d15b-4326-ac4a-ae77438d1bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:44.846233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9004-d15b-4326-ac4a-ae77438d1bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:44.846647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9004-d15b-4326-ac4a-ae77438d1bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:44.847752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec9004-d15b-4326-ac4a-ae77438d1bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:22:44.848415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec9004-d15b-4326-ac4a-ae77438d1bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:44.850223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec9004-d15b-4326-ac4a-ae77438d1bf4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:44.867760 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1343/2681] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:22:44.869793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a463bc1-2280-482c-b20f-6431e8f16d11 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:44.882981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a463bc1-2280-482c-b20f-6431e8f16d11 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:44.888113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b573e84-2703-4e47-8236-81fbe34b1e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:44.888793 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b573e84-2703-4e47-8236-81fbe34b1e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:44.889081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b573e84-2703-4e47-8236-81fbe34b1e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:44.889364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b573e84-2703-4e47-8236-81fbe34b1e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:44.890614 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b573e84-2703-4e47-8236-81fbe34b1e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:44.892587 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b573e84-2703-4e47-8236-81fbe34b1e56 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:44.923827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1339/2682] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:44 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:44.929060 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9004-d15b-4326-ac4a-ae77438d1bf4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:44.943633 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9004-d15b-4326-ac4a-ae77438d1bf4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:45.039401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41057-ff15-43ca-96ee-5c38885226bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:45.040087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41057-ff15-43ca-96ee-5c38885226bf admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:45.040378 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41057-ff15-43ca-96ee-5c38885226bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:45.040650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41057-ff15-43ca-96ee-5c38885226bf admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:45.041358 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41057-ff15-43ca-96ee-5c38885226bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:45.043478 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41057-ff15-43ca-96ee-5c38885226bf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:45.065755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1344/2683] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:44 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:45.070272 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b573e84-2703-4e47-8236-81fbe34b1e56 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:45.083595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b573e84-2703-4e47-8236-81fbe34b1e56 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:45.092058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c0bf7-9986-4068-a4fd-41c9b9dae904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:45.093698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c0bf7-9986-4068-a4fd-41c9b9dae904 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:45.093698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c0bf7-9986-4068-a4fd-41c9b9dae904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:45.093698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c0bf7-9986-4068-a4fd-41c9b9dae904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:45.094968 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c0bf7-9986-4068-a4fd-41c9b9dae904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:45.096970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c0bf7-9986-4068-a4fd-41c9b9dae904 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:45.108930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1340/2684] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:45.112905 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41057-ff15-43ca-96ee-5c38885226bf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:45.125412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41057-ff15-43ca-96ee-5c38885226bf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:45.132408 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47284a-b50c-4672-9a7d-e59d05c1aca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:45.134309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47284a-b50c-4672-9a7d-e59d05c1aca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:45.134309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47284a-b50c-4672-9a7d-e59d05c1aca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:45.134309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47284a-b50c-4672-9a7d-e59d05c1aca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:45.134309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47284a-b50c-4672-9a7d-e59d05c1aca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:45.135774 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47284a-b50c-4672-9a7d-e59d05c1aca0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:45.150194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1345/2685] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:22:45.155132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c0bf7-9986-4068-a4fd-41c9b9dae904 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:45.166214 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c0bf7-9986-4068-a4fd-41c9b9dae904 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:45.171391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018a00c-4477-497e-93f9-5c13be74e3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:45.172402 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018a00c-4477-497e-93f9-5c13be74e3ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:45.172768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018a00c-4477-497e-93f9-5c13be74e3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:45.173118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018a00c-4477-497e-93f9-5c13be74e3ff admin admin] PATH_INFO: `/v3/projects/f756b95ab7da4271a50fa3b1a5276eea/users/00ab892a9233421886b849b879bd8a7e/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:45.189484 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018a00c-4477-497e-93f9-5c13be74e3ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f756b95ab7da4271a50fa3b1a5276eea, user_id=00ab892a9233421886b849b879bd8a7e, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:45.192781 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018a00c-4477-497e-93f9-5c13be74e3ff admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:45.219762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1341/2686] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:45 2022] PUT /identity/v3/projects/f756b95ab7da4271a50fa3b1a5276eea/users/00ab892a9233421886b849b879bd8a7e/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:45.224671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d4b43-0ed4-4ee5-81bb-771241e9823d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:45.224976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d4b43-0ed4-4ee5-81bb-771241e9823d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:45.225265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d4b43-0ed4-4ee5-81bb-771241e9823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:45.247977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-340d4b43-0ed4-4ee5-81bb-771241e9823d None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:45.250132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-340d4b43-0ed4-4ee5-81bb-771241e9823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:45.260677 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:45.323958 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1346/2687] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:45.333574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018a00c-4477-497e-93f9-5c13be74e3ff admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:45.335396 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6018a00c-4477-497e-93f9-5c13be74e3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:45.406162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018a00c-4477-497e-93f9-5c13be74e3ff admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:45.411126 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abccb3a-c3e6-4ebd-bda7-8fb265caaced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:45.412096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abccb3a-c3e6-4ebd-bda7-8fb265caaced service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:45.412693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abccb3a-c3e6-4ebd-bda7-8fb265caaced service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:45.413248 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abccb3a-c3e6-4ebd-bda7-8fb265caaced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:45.430671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abccb3a-c3e6-4ebd-bda7-8fb265caaced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:45.432322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abccb3a-c3e6-4ebd-bda7-8fb265caaced service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:45.447105 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1342/2688] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:46.301042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeef3ad-1d67-4c29-b735-476e20a552ff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:46.301609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeef3ad-1d67-4c29-b735-476e20a552ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:46.301722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeef3ad-1d67-4c29-b735-476e20a552ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:46.334265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0aeef3ad-1d67-4c29-b735-476e20a552ff None None] MFA Rules not processed for user `00ab892a9233421886b849b879bd8a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:46.335675 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0aeef3ad-1d67-4c29-b735-476e20a552ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:46.416284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1347/2689] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:46.428460 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abccb3a-c3e6-4ebd-bda7-8fb265caaced service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:46.430015 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9abccb3a-c3e6-4ebd-bda7-8fb265caaced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:46.459727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:46.516158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abccb3a-c3e6-4ebd-bda7-8fb265caaced service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:46.523730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed9406-92c2-4277-ba42-990ee13a1dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:46.524392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed9406-92c2-4277-ba42-990ee13a1dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:46.524681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed9406-92c2-4277-ba42-990ee13a1dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:46.524957 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed9406-92c2-4277-ba42-990ee13a1dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:46.535890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed9406-92c2-4277-ba42-990ee13a1dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:46.538071 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed9406-92c2-4277-ba42-990ee13a1dc7 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:46.549177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1343/2690] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:48.019979 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e75f-7c3a-45bb-a105-d2fec9080f77 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:48.020526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e75f-7c3a-45bb-a105-d2fec9080f77 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:48.020526 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e75f-7c3a-45bb-a105-d2fec9080f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:48.039326 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4b52e75f-7c3a-45bb-a105-d2fec9080f77 None None] MFA Rules not processed for user `00ab892a9233421886b849b879bd8a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:48.040665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b52e75f-7c3a-45bb-a105-d2fec9080f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:48.048667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1348/2691] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:50.019537 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed9406-92c2-4277-ba42-990ee13a1dc7 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:50.031250 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed9406-92c2-4277-ba42-990ee13a1dc7 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:50.036871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3412-1c51-4e1a-969e-6a7d9db1f708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:50.038247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3412-1c51-4e1a-969e-6a7d9db1f708 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:50.038247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3412-1c51-4e1a-969e-6a7d9db1f708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:50.038247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3412-1c51-4e1a-969e-6a7d9db1f708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:50.040945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-690d3412-1c51-4e1a-969e-6a7d9db1f708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:50.057029 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d3412-1c51-4e1a-969e-6a7d9db1f708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:50.058448 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d3412-1c51-4e1a-969e-6a7d9db1f708 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:50.070637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1344/2692] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:54.016079 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52e75f-7c3a-45bb-a105-d2fec9080f77 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:54.017653 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b52e75f-7c3a-45bb-a105-d2fec9080f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:54.033269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52e75f-7c3a-45bb-a105-d2fec9080f77 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:54.038064 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:54.038949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:54.039228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:54.039577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] PATH_INFO: `/v3/users/99798aca08904bc787f5d0afee8d0ac7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:54.040772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99798aca08904bc787f5d0afee8d0ac7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:54.042697 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:54.095359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99798aca08904bc787f5d0afee8d0ac7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.104508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99798aca08904bc787f5d0afee8d0ac7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.113810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99798aca08904bc787f5d0afee8d0ac7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.126261 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99798aca08904bc787f5d0afee8d0ac7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.126882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-19ea2209-a901-4cf0-8ced-85a9fc657d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99798aca08904bc787f5d0afee8d0ac7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.133199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1349/2693] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:54 2022] DELETE /identity/v3/users/99798aca08904bc787f5d0afee8d0ac7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:54.315685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3412-1c51-4e1a-969e-6a7d9db1f708 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:54.333624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3412-1c51-4e1a-969e-6a7d9db1f708 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:54.421924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:54.423166 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:54.423490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:54.423769 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] PATH_INFO: `/v3/projects/b745569c01e04aa18f5494b0856bc3d1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:54.425437 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b745569c01e04aa18f5494b0856bc3d1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:54.427994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:54.487785 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b745569c01e04aa18f5494b0856bc3d1', 'request_id': 'req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.494154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b745569c01e04aa18f5494b0856bc3d1', 'request_id': 'req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.501016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b745569c01e04aa18f5494b0856bc3d1', 'request_id': 'req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.520404 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] The token cache is being invalidate because project b745569c01e04aa18f5494b0856bc3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:54.520704 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.521522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bfdf6a9b-f1fa-47b4-8f57-63da9bc45d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:54.523153 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1345/2694] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:54 2022] DELETE /identity/v3/projects/b745569c01e04aa18f5494b0856bc3d1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:54.528848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7121ba3-c62c-46f9-ae7d-3bf98de00674 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:54.529299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7121ba3-c62c-46f9-ae7d-3bf98de00674 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:54.530418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7121ba3-c62c-46f9-ae7d-3bf98de00674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:54.553321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f7121ba3-c62c-46f9-ae7d-3bf98de00674 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:54.554863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f7121ba3-c62c-46f9-ae7d-3bf98de00674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:54.560688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:54.645179 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1350/2695] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:54.653412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededec61-6d41-4ab1-8441-95e853d8e410 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:54.653628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededec61-6d41-4ab1-8441-95e853d8e410 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:54.653980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededec61-6d41-4ab1-8441-95e853d8e410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:54.677206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ededec61-6d41-4ab1-8441-95e853d8e410 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:54.678452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ededec61-6d41-4ab1-8441-95e853d8e410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:54.688656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1346/2696] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:54.693077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7121ba3-c62c-46f9-ae7d-3bf98de00674 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:54.704749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7121ba3-c62c-46f9-ae7d-3bf98de00674 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:54.710684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515ee77-b6f5-46f2-999a-34ec92e543e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:54.712247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515ee77-b6f5-46f2-999a-34ec92e543e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:54.713004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515ee77-b6f5-46f2-999a-34ec92e543e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:54.713479 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515ee77-b6f5-46f2-999a-34ec92e543e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:54.715077 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515ee77-b6f5-46f2-999a-34ec92e543e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:22:54.715610 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515ee77-b6f5-46f2-999a-34ec92e543e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:54.717431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515ee77-b6f5-46f2-999a-34ec92e543e9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:54.730964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1351/2697] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:22:54.735329 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededec61-6d41-4ab1-8441-95e853d8e410 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:54.745952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededec61-6d41-4ab1-8441-95e853d8e410 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:54.752708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e7676-92c0-4d5a-9b51-006e9f06df3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:54.753612 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e7676-92c0-4d5a-9b51-006e9f06df3c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:54.753905 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e7676-92c0-4d5a-9b51-006e9f06df3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:54.754187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e7676-92c0-4d5a-9b51-006e9f06df3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:54.754915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e7676-92c0-4d5a-9b51-006e9f06df3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:54.757006 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e7676-92c0-4d5a-9b51-006e9f06df3c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:54.791307 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1347/2698] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:54 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:54.793628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515ee77-b6f5-46f2-999a-34ec92e543e9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:54.804252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515ee77-b6f5-46f2-999a-34ec92e543e9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:54.890090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97c13e-c1ad-43a2-908d-f84fdb243323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:54.890090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97c13e-c1ad-43a2-908d-f84fdb243323 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:54.890090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97c13e-c1ad-43a2-908d-f84fdb243323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:54.890090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97c13e-c1ad-43a2-908d-f84fdb243323 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:54.890428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97c13e-c1ad-43a2-908d-f84fdb243323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:54.890428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97c13e-c1ad-43a2-908d-f84fdb243323 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:54.911302 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1352/2699] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:54 2022] POST /identity/v3/users => generated 302 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:54.913419 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e7676-92c0-4d5a-9b51-006e9f06df3c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:54.924031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e7676-92c0-4d5a-9b51-006e9f06df3c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:54.929828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e910253-49b6-4564-99ed-2fae696b2ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:54.930955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e910253-49b6-4564-99ed-2fae696b2ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:54.931530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e910253-49b6-4564-99ed-2fae696b2ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:54.931698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e910253-49b6-4564-99ed-2fae696b2ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:54.932497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e910253-49b6-4564-99ed-2fae696b2ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:54.934369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e910253-49b6-4564-99ed-2fae696b2ea4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:54.937150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:54.948118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1348/2700] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:54.951723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97c13e-c1ad-43a2-908d-f84fdb243323 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:54.962980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97c13e-c1ad-43a2-908d-f84fdb243323 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:54.968056 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6add7c-606c-4b6e-9ce2-183fe2a6eab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:54.968723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6add7c-606c-4b6e-9ce2-183fe2a6eab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:54.969014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6add7c-606c-4b6e-9ce2-183fe2a6eab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:54.969614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6add7c-606c-4b6e-9ce2-183fe2a6eab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:54.970040 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6add7c-606c-4b6e-9ce2-183fe2a6eab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:54.971721 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6add7c-606c-4b6e-9ce2-183fe2a6eab2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:54.985812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1353/2701] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:55.001940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e910253-49b6-4564-99ed-2fae696b2ea4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:55.013557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e910253-49b6-4564-99ed-2fae696b2ea4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:55.020537 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7d488-ae14-4b8f-baad-59b0a3469ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:55.021618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7d488-ae14-4b8f-baad-59b0a3469ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.022026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7d488-ae14-4b8f-baad-59b0a3469ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.022442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7d488-ae14-4b8f-baad-59b0a3469ba1 admin admin] PATH_INFO: `/v3/projects/28968f10734e48868ac9522e6ad7ac84/users/2ef739cea32c43cc989632406334520c/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.039520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7d488-ae14-4b8f-baad-59b0a3469ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28968f10734e48868ac9522e6ad7ac84, user_id=2ef739cea32c43cc989632406334520c, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:55.042497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7d488-ae14-4b8f-baad-59b0a3469ba1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:55.067506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1349/2702] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:55 2022] PUT /identity/v3/projects/28968f10734e48868ac9522e6ad7ac84/users/2ef739cea32c43cc989632406334520c/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:55.070324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdf146-2f29-4958-865a-19ff21491522 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.070761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdf146-2f29-4958-865a-19ff21491522 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.070910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdf146-2f29-4958-865a-19ff21491522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.100373 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-72cdf146-2f29-4958-865a-19ff21491522 None None] MFA Rules not processed for user `2ef739cea32c43cc989632406334520c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:55.102126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-72cdf146-2f29-4958-865a-19ff21491522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:55.179744 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1354/2703] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:55.184658 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7d488-ae14-4b8f-baad-59b0a3469ba1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:55.246833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7d488-ae14-4b8f-baad-59b0a3469ba1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:55.252611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1588a8-fbc8-47ed-b982-4238695cc019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:55.253274 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1588a8-fbc8-47ed-b982-4238695cc019 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.253567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1588a8-fbc8-47ed-b982-4238695cc019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.253852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1588a8-fbc8-47ed-b982-4238695cc019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.254542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1588a8-fbc8-47ed-b982-4238695cc019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:55.256468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1588a8-fbc8-47ed-b982-4238695cc019 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:55.282205 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1350/2704] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:55 2022] POST /identity/v3/projects => generated 403 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:55.286471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdf146-2f29-4958-865a-19ff21491522 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:55.293746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:55.298939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdf146-2f29-4958-865a-19ff21491522 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:55.368282 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32b476-f084-40df-88a5-0f78bb8d3e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:55.368966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32b476-f084-40df-88a5-0f78bb8d3e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.369255 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32b476-f084-40df-88a5-0f78bb8d3e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.369536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32b476-f084-40df-88a5-0f78bb8d3e52 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.370410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32b476-f084-40df-88a5-0f78bb8d3e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:55.372428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32b476-f084-40df-88a5-0f78bb8d3e52 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:55.394237 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1355/2705] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:55 2022] POST /identity/v3/users => generated 308 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:22:55.398353 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1588a8-fbc8-47ed-b982-4238695cc019 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:55.409767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1588a8-fbc8-47ed-b982-4238695cc019 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:55.414621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a81bf-bd6f-48c6-98d7-c731c9eb8d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:55.415326 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a81bf-bd6f-48c6-98d7-c731c9eb8d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.415634 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a81bf-bd6f-48c6-98d7-c731c9eb8d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.415955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a81bf-bd6f-48c6-98d7-c731c9eb8d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.416703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a81bf-bd6f-48c6-98d7-c731c9eb8d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:55.418317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a81bf-bd6f-48c6-98d7-c731c9eb8d82 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:55.431529 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1351/2706] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:55.436266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32b476-f084-40df-88a5-0f78bb8d3e52 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:55.447607 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32b476-f084-40df-88a5-0f78bb8d3e52 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:55.452352 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e1dfb-2bc8-4331-b070-304322679b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:55.453218 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e1dfb-2bc8-4331-b070-304322679b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.453504 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e1dfb-2bc8-4331-b070-304322679b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.453784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e1dfb-2bc8-4331-b070-304322679b34 admin admin] PATH_INFO: `/v3/projects/6e1218e8ff9b4db4a822ce23b429ab90/users/00b9b60224ae4539993e0831cf5e799f/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.470634 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e1dfb-2bc8-4331-b070-304322679b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1218e8ff9b4db4a822ce23b429ab90, user_id=00b9b60224ae4539993e0831cf5e799f, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:55.473344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e1dfb-2bc8-4331-b070-304322679b34 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:55.488650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1356/2707] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:55 2022] PUT /identity/v3/projects/6e1218e8ff9b4db4a822ce23b429ab90/users/00b9b60224ae4539993e0831cf5e799f/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:55.493169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a81bf-bd6f-48c6-98d7-c731c9eb8d82 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:55.504159 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a81bf-bd6f-48c6-98d7-c731c9eb8d82 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:55.561985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732d75a-e0f0-4d28-852a-7e21966c3ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:55.562660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732d75a-e0f0-4d28-852a-7e21966c3ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.562956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732d75a-e0f0-4d28-852a-7e21966c3ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.563234 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732d75a-e0f0-4d28-852a-7e21966c3ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.564002 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732d75a-e0f0-4d28-852a-7e21966c3ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:55.565630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732d75a-e0f0-4d28-852a-7e21966c3ec6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:55.575951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1352/2708] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:55.580266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e1dfb-2bc8-4331-b070-304322679b34 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:55.590718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e1dfb-2bc8-4331-b070-304322679b34 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:55.595992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febccfb-7dff-4a66-8930-7f630a925d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:55.596901 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febccfb-7dff-4a66-8930-7f630a925d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.597194 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febccfb-7dff-4a66-8930-7f630a925d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.597639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febccfb-7dff-4a66-8930-7f630a925d44 admin admin] PATH_INFO: `/v3/projects/6e1218e8ff9b4db4a822ce23b429ab90/users/00b9b60224ae4539993e0831cf5e799f/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.599751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febccfb-7dff-4a66-8930-7f630a925d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1218e8ff9b4db4a822ce23b429ab90, user_id=00b9b60224ae4539993e0831cf5e799f, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:55.602426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febccfb-7dff-4a66-8930-7f630a925d44 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:55.615019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1357/2709] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:55 2022] PUT /identity/v3/projects/6e1218e8ff9b4db4a822ce23b429ab90/users/00b9b60224ae4539993e0831cf5e799f/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:55.625567 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e46f1-1649-4ba5-bcfc-8ae170397064 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.625864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e46f1-1649-4ba5-bcfc-8ae170397064 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.626861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e46f1-1649-4ba5-bcfc-8ae170397064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.654380 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-508e46f1-1649-4ba5-bcfc-8ae170397064 None None] MFA Rules not processed for user `00b9b60224ae4539993e0831cf5e799f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:55.656056 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-508e46f1-1649-4ba5-bcfc-8ae170397064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:55.689737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:55.727959 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1353/2710] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:55.740738 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febccfb-7dff-4a66-8930-7f630a925d44 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:55.742439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0febccfb-7dff-4a66-8930-7f630a925d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:55.802412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febccfb-7dff-4a66-8930-7f630a925d44 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:55.806050 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2a173-2553-4abd-8277-48505bbd9e64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:55.806701 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2a173-2553-4abd-8277-48505bbd9e64 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:55.806988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2a173-2553-4abd-8277-48505bbd9e64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:55.807283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2a173-2553-4abd-8277-48505bbd9e64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:55.817692 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2a173-2553-4abd-8277-48505bbd9e64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:55.819245 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2a173-2553-4abd-8277-48505bbd9e64 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:55.830019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1358/2711] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:55 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:57.020315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e46f1-1649-4ba5-bcfc-8ae170397064 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:57.091219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e46f1-1649-4ba5-bcfc-8ae170397064 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:57.097443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:57.098343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:57.098664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:57.098951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] PATH_INFO: `/v3/users/2ef739cea32c43cc989632406334520c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:57.100264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef739cea32c43cc989632406334520c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:57.102213 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:57.163370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef739cea32c43cc989632406334520c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.164049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef739cea32c43cc989632406334520c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.177370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef739cea32c43cc989632406334520c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.192299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef739cea32c43cc989632406334520c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.200670 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef739cea32c43cc989632406334520c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.207952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1354/2712] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:57 2022] DELETE /identity/v3/users/2ef739cea32c43cc989632406334520c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:57.212571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452797c-fbf6-44e3-939b-c1c14dfa8d13 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:57.213026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452797c-fbf6-44e3-939b-c1c14dfa8d13 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:57.213139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452797c-fbf6-44e3-939b-c1c14dfa8d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:57.233835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3452797c-fbf6-44e3-939b-c1c14dfa8d13 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:57.235206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3452797c-fbf6-44e3-939b-c1c14dfa8d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:57.294228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:57.303964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1359/2713] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:57.313871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:57.315355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:57.371486 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:57.375772 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c2c43-d397-42b8-93c0-950a38325ae5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:57.380529 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c58e53-a2d8-44c5-87c3-2945d331ffb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:57.381611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c58e53-a2d8-44c5-87c3-2945d331ffb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:57.382007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c58e53-a2d8-44c5-87c3-2945d331ffb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:57.382317 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c58e53-a2d8-44c5-87c3-2945d331ffb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:57.392915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c58e53-a2d8-44c5-87c3-2945d331ffb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:57.394374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c58e53-a2d8-44c5-87c3-2945d331ffb9 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:57.404431 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1355/2714] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:22:57.686945 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452797c-fbf6-44e3-939b-c1c14dfa8d13 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:57.688427 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3452797c-fbf6-44e3-939b-c1c14dfa8d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:57.703815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452797c-fbf6-44e3-939b-c1c14dfa8d13 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:57.708651 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:57.709568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:57.709838 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:57.710121 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] PATH_INFO: `/v3/projects/28968f10734e48868ac9522e6ad7ac84` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:57.711389 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] RBAC: Authorizing `identity:delete_project(project_id=28968f10734e48868ac9522e6ad7ac84)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:57.713341 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:57.762980 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28968f10734e48868ac9522e6ad7ac84', 'request_id': 'req-fa3d4745-c21b-4382-b72a-4fef3af6a86d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.769745 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28968f10734e48868ac9522e6ad7ac84', 'request_id': 'req-fa3d4745-c21b-4382-b72a-4fef3af6a86d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.775439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28968f10734e48868ac9522e6ad7ac84', 'request_id': 'req-fa3d4745-c21b-4382-b72a-4fef3af6a86d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.787995 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] The token cache is being invalidate because project 28968f10734e48868ac9522e6ad7ac84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:57.788424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.789075 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.790640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1360/2715] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:57 2022] DELETE /identity/v3/projects/28968f10734e48868ac9522e6ad7ac84 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:57.794721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c58e53-a2d8-44c5-87c3-2945d331ffb9 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:57.804703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c58e53-a2d8-44c5-87c3-2945d331ffb9 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:57.867251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:57.868197 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:57.868508 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:57.868801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] PATH_INFO: `/v3/users/00b9b60224ae4539993e0831cf5e799f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:57.870111 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b9b60224ae4539993e0831cf5e799f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:57.872091 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:57.932707 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b9b60224ae4539993e0831cf5e799f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.933415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b9b60224ae4539993e0831cf5e799f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.944429 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b9b60224ae4539993e0831cf5e799f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.960536 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b9b60224ae4539993e0831cf5e799f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.968744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3c1d820b-7f34-424c-97fe-895586906f12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b9b60224ae4539993e0831cf5e799f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:57.977464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1356/2716] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:22:57 2022] DELETE /identity/v3/users/00b9b60224ae4539993e0831cf5e799f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:58.329308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:58.331647 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:58.350989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d4745-c21b-4382-b72a-4fef3af6a86d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:58.437462 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:58.438386 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:58.438665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:58.438937 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] PATH_INFO: `/v3/projects/6e1218e8ff9b4db4a822ce23b429ab90` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:58.440281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1218e8ff9b4db4a822ce23b429ab90)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:58.442310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:58.511977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1218e8ff9b4db4a822ce23b429ab90', 'request_id': 'req-f1cfb50c-fba5-475a-9677-98c537466e4d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:58.518543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1218e8ff9b4db4a822ce23b429ab90', 'request_id': 'req-f1cfb50c-fba5-475a-9677-98c537466e4d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:58.524702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1218e8ff9b4db4a822ce23b429ab90', 'request_id': 'req-f1cfb50c-fba5-475a-9677-98c537466e4d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:58.542330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] The token cache is being invalidate because project 6e1218e8ff9b4db4a822ce23b429ab90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:22:58.542619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:58.543415 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f1cfb50c-fba5-475a-9677-98c537466e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:22:58.545092 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1361/2717] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:22:58 2022] DELETE /identity/v3/projects/6e1218e8ff9b4db4a822ce23b429ab90 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:58.550735 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6417c5-955a-4f9e-80fb-29b44b6b5ff3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:58.551333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6417c5-955a-4f9e-80fb-29b44b6b5ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:58.551333 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6417c5-955a-4f9e-80fb-29b44b6b5ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:58.582240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4a6417c5-955a-4f9e-80fb-29b44b6b5ff3 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:58.583414 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4a6417c5-955a-4f9e-80fb-29b44b6b5ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:58.648809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1357/2718] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:58.658506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcf507-6b8b-473b-ae49-3381286833e4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:58.659058 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcf507-6b8b-473b-ae49-3381286833e4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:58.659384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcf507-6b8b-473b-ae49-3381286833e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:58.679334 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-84fcf507-6b8b-473b-ae49-3381286833e4 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:58.680594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-84fcf507-6b8b-473b-ae49-3381286833e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:58.688977 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1362/2719] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:58.693391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6417c5-955a-4f9e-80fb-29b44b6b5ff3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:58.706347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6417c5-955a-4f9e-80fb-29b44b6b5ff3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:58.711236 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038710e4-f68e-4f6a-9c8f-b1ee471b4e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:58.711934 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038710e4-f68e-4f6a-9c8f-b1ee471b4e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:58.712221 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038710e4-f68e-4f6a-9c8f-b1ee471b4e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:58.712502 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038710e4-f68e-4f6a-9c8f-b1ee471b4e07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:58.713238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038710e4-f68e-4f6a-9c8f-b1ee471b4e07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:22:58.713571 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038710e4-f68e-4f6a-9c8f-b1ee471b4e07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:58.715403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038710e4-f68e-4f6a-9c8f-b1ee471b4e07 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:58.723237 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:58.731413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1358/2720] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:22:58.735387 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcf507-6b8b-473b-ae49-3381286833e4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:58.745457 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcf507-6b8b-473b-ae49-3381286833e4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:58.750532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04974f9e-2299-420d-9b90-f74b9a7dc87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:58.751213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974f9e-2299-420d-9b90-f74b9a7dc87e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:58.751543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974f9e-2299-420d-9b90-f74b9a7dc87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:58.751829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974f9e-2299-420d-9b90-f74b9a7dc87e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:58.752543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04974f9e-2299-420d-9b90-f74b9a7dc87e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:58.754482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04974f9e-2299-420d-9b90-f74b9a7dc87e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:58.758100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:22:58.794755 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1363/2721] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:22:58 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:58.796338 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038710e4-f68e-4f6a-9c8f-b1ee471b4e07 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:58.806495 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038710e4-f68e-4f6a-9c8f-b1ee471b4e07 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:58.887767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ba185-0942-46b5-be90-f64630228487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:58.888379 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ba185-0942-46b5-be90-f64630228487 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:58.888664 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ba185-0942-46b5-be90-f64630228487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:58.888943 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ba185-0942-46b5-be90-f64630228487 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:58.892044 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ba185-0942-46b5-be90-f64630228487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:58.894375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ba185-0942-46b5-be90-f64630228487 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:58.917645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1359/2722] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:22:58 2022] POST /identity/v3/users => generated 303 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:22:58.922080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04974f9e-2299-420d-9b90-f74b9a7dc87e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:58.933112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04974f9e-2299-420d-9b90-f74b9a7dc87e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:58.938291 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb825a-c25f-4bbd-b89a-37556abb445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:58.939028 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb825a-c25f-4bbd-b89a-37556abb445f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:58.939354 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb825a-c25f-4bbd-b89a-37556abb445f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:58.939644 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb825a-c25f-4bbd-b89a-37556abb445f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:58.940384 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb825a-c25f-4bbd-b89a-37556abb445f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:58.942008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb825a-c25f-4bbd-b89a-37556abb445f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:58.953622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1364/2723] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:58.958476 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ba185-0942-46b5-be90-f64630228487 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:58.969981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ba185-0942-46b5-be90-f64630228487 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:58.974809 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210a2c4-1acb-4958-b69b-04d1bb3abfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:58.975630 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210a2c4-1acb-4958-b69b-04d1bb3abfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:58.975970 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210a2c4-1acb-4958-b69b-04d1bb3abfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:58.976267 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210a2c4-1acb-4958-b69b-04d1bb3abfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:58.977016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210a2c4-1acb-4958-b69b-04d1bb3abfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:58.978646 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210a2c4-1acb-4958-b69b-04d1bb3abfb1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:59.007025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1360/2724] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:22:59.014483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb825a-c25f-4bbd-b89a-37556abb445f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:59.052864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb825a-c25f-4bbd-b89a-37556abb445f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:59.060595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24786134-9746-42eb-99df-b14133d0f54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:59.062007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24786134-9746-42eb-99df-b14133d0f54d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:59.062439 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24786134-9746-42eb-99df-b14133d0f54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:59.062724 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24786134-9746-42eb-99df-b14133d0f54d admin admin] PATH_INFO: `/v3/projects/a654962d670645aba4fe68629c84c9ef/users/e2c47823c61c4e31855e61d9d73ccf3b/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:59.087959 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24786134-9746-42eb-99df-b14133d0f54d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a654962d670645aba4fe68629c84c9ef, user_id=e2c47823c61c4e31855e61d9d73ccf3b, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:59.091100 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24786134-9746-42eb-99df-b14133d0f54d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:59.110330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1365/2725] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:22:59 2022] PUT /identity/v3/projects/a654962d670645aba4fe68629c84c9ef/users/e2c47823c61c4e31855e61d9d73ccf3b/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:22:59.116328 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70829fed-f94f-4124-bfdb-0d08082cd025 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:59.116624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70829fed-f94f-4124-bfdb-0d08082cd025 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:59.116907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70829fed-f94f-4124-bfdb-0d08082cd025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:59.118068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbbba5-8008-46da-b9d4-e432039bceec None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:59.118563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbbba5-8008-46da-b9d4-e432039bceec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:59.118954 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbbba5-8008-46da-b9d4-e432039bceec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:59.134824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-70829fed-f94f-4124-bfdb-0d08082cd025 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:59.136150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-70829fed-f94f-4124-bfdb-0d08082cd025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:59.139534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a0bbbba5-8008-46da-b9d4-e432039bceec None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:22:59.140925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a0bbbba5-8008-46da-b9d4-e432039bceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:59.204704 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1366/2726] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Jul 22 11:22:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:59.207186 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1361/2727] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:22:59.216776 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbbba5-8008-46da-b9d4-e432039bceec None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:59.218280 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a0bbbba5-8008-46da-b9d4-e432039bceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:59.220058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70829fed-f94f-4124-bfdb-0d08082cd025 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:22:59.222608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-70829fed-f94f-4124-bfdb-0d08082cd025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:22:59.288179 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70829fed-f94f-4124-bfdb-0d08082cd025 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:59.292868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cff747-0140-460a-831a-368029f6e60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:59.293725 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cff747-0140-460a-831a-368029f6e60d service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:59.293836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cff747-0140-460a-831a-368029f6e60d service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:59.294075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cff747-0140-460a-831a-368029f6e60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:59.305475 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cff747-0140-460a-831a-368029f6e60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:59.306937 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cff747-0140-460a-831a-368029f6e60d service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:59.313993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbbba5-8008-46da-b9d4-e432039bceec None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:22:59.318637 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1362/2728] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:22:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:22:59.319328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70cab0-c16e-4045-a890-5bd61ca61388 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:22:59.320018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70cab0-c16e-4045-a890-5bd61ca61388 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:22:59.320394 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70cab0-c16e-4045-a890-5bd61ca61388 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:22:59.320678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70cab0-c16e-4045-a890-5bd61ca61388 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:22:59.332546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70cab0-c16e-4045-a890-5bd61ca61388 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:22:59.334228 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70cab0-c16e-4045-a890-5bd61ca61388 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:22:59.343297 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1367/2729] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:22:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 22 11:23:00.433699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ccd66-2956-4323-8d6c-7138fe74c197 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:00.434694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ccd66-2956-4323-8d6c-7138fe74c197 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:00.434694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ccd66-2956-4323-8d6c-7138fe74c197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:00.480035 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-548ccd66-2956-4323-8d6c-7138fe74c197 None None] MFA Rules not processed for user `e2c47823c61c4e31855e61d9d73ccf3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:00.481897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-548ccd66-2956-4323-8d6c-7138fe74c197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:00.496090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:00.639648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1363/2730] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:00.644144 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70cab0-c16e-4045-a890-5bd61ca61388 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:00.665374 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70cab0-c16e-4045-a890-5bd61ca61388 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:00.684863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81964c-b9d6-49cd-ad36-8f6c60f6de29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:00.686516 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81964c-b9d6-49cd-ad36-8f6c60f6de29 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:00.687436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81964c-b9d6-49cd-ad36-8f6c60f6de29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:00.688096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81964c-b9d6-49cd-ad36-8f6c60f6de29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:00.690538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81964c-b9d6-49cd-ad36-8f6c60f6de29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:00.698021 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81964c-b9d6-49cd-ad36-8f6c60f6de29 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:00.744165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1368/2731] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:00 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:00.748506 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ccd66-2956-4323-8d6c-7138fe74c197 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:00.763520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ccd66-2956-4323-8d6c-7138fe74c197 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:00.849881 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d62c5-f66b-4556-8412-d6e41e420cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:00.851390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d62c5-f66b-4556-8412-d6e41e420cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:00.852062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d62c5-f66b-4556-8412-d6e41e420cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:00.852693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d62c5-f66b-4556-8412-d6e41e420cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:00.853989 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d62c5-f66b-4556-8412-d6e41e420cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:00.857277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d62c5-f66b-4556-8412-d6e41e420cd4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:00.891724 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1364/2732] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:00 2022] POST /identity/v3/users => generated 308 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:23:00.896528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81964c-b9d6-49cd-ad36-8f6c60f6de29 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:00.923189 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81964c-b9d6-49cd-ad36-8f6c60f6de29 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:00.930428 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88abbe-c5b2-4fbc-ad4d-c0eb6663009c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:00.931586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88abbe-c5b2-4fbc-ad4d-c0eb6663009c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:00.932044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88abbe-c5b2-4fbc-ad4d-c0eb6663009c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:00.932951 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88abbe-c5b2-4fbc-ad4d-c0eb6663009c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:00.934837 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88abbe-c5b2-4fbc-ad4d-c0eb6663009c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:00.939671 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88abbe-c5b2-4fbc-ad4d-c0eb6663009c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:00.962483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1369/2733] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:23:00.968705 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d62c5-f66b-4556-8412-d6e41e420cd4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:00.983971 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d62c5-f66b-4556-8412-d6e41e420cd4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:00.991057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcb046-e9b9-4f2a-a601-f4f4dbaa4ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:00.992051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcb046-e9b9-4f2a-a601-f4f4dbaa4ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:00.992366 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcb046-e9b9-4f2a-a601-f4f4dbaa4ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:00.992647 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcb046-e9b9-4f2a-a601-f4f4dbaa4ccf admin admin] PATH_INFO: `/v3/projects/50c08341b4514b148a60922002cf3593/users/bfc3574b8eaf4b26b509c97976180628/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:01.010766 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcb046-e9b9-4f2a-a601-f4f4dbaa4ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c08341b4514b148a60922002cf3593, user_id=bfc3574b8eaf4b26b509c97976180628, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:01.014434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcb046-e9b9-4f2a-a601-f4f4dbaa4ccf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:01.033554 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1365/2734] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:00 2022] PUT /identity/v3/projects/50c08341b4514b148a60922002cf3593/users/bfc3574b8eaf4b26b509c97976180628/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:01.038269 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88abbe-c5b2-4fbc-ad4d-c0eb6663009c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:01.085988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88abbe-c5b2-4fbc-ad4d-c0eb6663009c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:01.095751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:01.201844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb3e73-f17f-4d6e-bf25-dc103d4b70a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:01.202521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb3e73-f17f-4d6e-bf25-dc103d4b70a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:01.202812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb3e73-f17f-4d6e-bf25-dc103d4b70a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:01.203097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb3e73-f17f-4d6e-bf25-dc103d4b70a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:01.203867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb3e73-f17f-4d6e-bf25-dc103d4b70a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:01.207347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb3e73-f17f-4d6e-bf25-dc103d4b70a7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:01.227220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1370/2735] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:01.228471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcb046-e9b9-4f2a-a601-f4f4dbaa4ccf admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:01.244301 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcb046-e9b9-4f2a-a601-f4f4dbaa4ccf admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:01.251488 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe3fe0-60c2-4eb0-948c-2b9faf33c107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:01.252892 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe3fe0-60c2-4eb0-948c-2b9faf33c107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:01.253650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe3fe0-60c2-4eb0-948c-2b9faf33c107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:01.254219 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe3fe0-60c2-4eb0-948c-2b9faf33c107 admin admin] PATH_INFO: `/v3/projects/50c08341b4514b148a60922002cf3593/users/bfc3574b8eaf4b26b509c97976180628/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:01.257141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe3fe0-60c2-4eb0-948c-2b9faf33c107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c08341b4514b148a60922002cf3593, user_id=bfc3574b8eaf4b26b509c97976180628, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:01.260057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe3fe0-60c2-4eb0-948c-2b9faf33c107 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:01.279428 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1366/2736] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:01 2022] PUT /identity/v3/projects/50c08341b4514b148a60922002cf3593/users/bfc3574b8eaf4b26b509c97976180628/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:02.232603 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7dd178-4d0d-40f6-8a71-e99901b41057 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:02.233142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7dd178-4d0d-40f6-8a71-e99901b41057 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:02.233281 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7dd178-4d0d-40f6-8a71-e99901b41057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:02.278336 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ac7dd178-4d0d-40f6-8a71-e99901b41057 None None] MFA Rules not processed for user `bfc3574b8eaf4b26b509c97976180628`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:02.279608 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ac7dd178-4d0d-40f6-8a71-e99901b41057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:02.380789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1371/2737] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:03.876519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe3fe0-60c2-4eb0-948c-2b9faf33c107 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:04.000417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe3fe0-60c2-4eb0-948c-2b9faf33c107 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:04.009040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:04.010964 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:04.012077 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:04.012680 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] PATH_INFO: `/v3/users/e2c47823c61c4e31855e61d9d73ccf3b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:04.016434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2c47823c61c4e31855e61d9d73ccf3b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:04.021056 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:04.025589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:04.105660 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2c47823c61c4e31855e61d9d73ccf3b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.106443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2c47823c61c4e31855e61d9d73ccf3b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.137342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2c47823c61c4e31855e61d9d73ccf3b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.166681 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2c47823c61c4e31855e61d9d73ccf3b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.182068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2c47823c61c4e31855e61d9d73ccf3b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.197287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1367/2738] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:03 2022] DELETE /identity/v3/users/e2c47823c61c4e31855e61d9d73ccf3b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:04.528025 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7dd178-4d0d-40f6-8a71-e99901b41057 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:04.553423 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7dd178-4d0d-40f6-8a71-e99901b41057 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:04.637220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:04.638459 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:04.639020 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:04.639577 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] PATH_INFO: `/v3/projects/a654962d670645aba4fe68629c84c9ef` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:04.640930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a654962d670645aba4fe68629c84c9ef)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:04.642952 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:04.693737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:04.711067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a654962d670645aba4fe68629c84c9ef', 'request_id': 'req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.719798 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a654962d670645aba4fe68629c84c9ef', 'request_id': 'req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.729576 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a654962d670645aba4fe68629c84c9ef', 'request_id': 'req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.749509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] The token cache is being invalidate because project a654962d670645aba4fe68629c84c9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:04.749869 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.751030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.752860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1372/2739] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:04 2022] DELETE /identity/v3/projects/a654962d670645aba4fe68629c84c9ef => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:04.772329 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:04.774257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:04.793965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cda55f-e5c3-4488-ac0c-2e2a9e3db09b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:04.875138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:04.876412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:04.876701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:04.877254 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] PATH_INFO: `/v3/users/bfc3574b8eaf4b26b509c97976180628` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:04.878411 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc3574b8eaf4b26b509c97976180628)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:04.880978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:04.945313 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc3574b8eaf4b26b509c97976180628'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.946026 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc3574b8eaf4b26b509c97976180628'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.960769 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc3574b8eaf4b26b509c97976180628'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.979888 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc3574b8eaf4b26b509c97976180628'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.990711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53f2620b-1abc-4ffd-ae7c-d28cdb0e220c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc3574b8eaf4b26b509c97976180628'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:04.993112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:05.005788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1368/2740] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:04 2022] DELETE /identity/v3/users/bfc3574b8eaf4b26b509c97976180628 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:05.282547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:05.284711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:05.309324 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbeb8c-5a68-4a9b-8ae5-33b7884d5629 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:05.412309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:05.413211 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:05.413524 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:05.413756 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] PATH_INFO: `/v3/projects/50c08341b4514b148a60922002cf3593` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:05.415044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c08341b4514b148a60922002cf3593)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:05.417086 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:05.484367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c08341b4514b148a60922002cf3593', 'request_id': 'req-9c3c706f-d12b-48f4-9b64-ae47e9b23061'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:05.493494 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c08341b4514b148a60922002cf3593', 'request_id': 'req-9c3c706f-d12b-48f4-9b64-ae47e9b23061'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:05.502137 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c08341b4514b148a60922002cf3593', 'request_id': 'req-9c3c706f-d12b-48f4-9b64-ae47e9b23061'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:05.519292 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] The token cache is being invalidate because project 50c08341b4514b148a60922002cf3593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:05.519550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:05.520184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3c706f-d12b-48f4-9b64-ae47e9b23061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:05.521841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1373/2741] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:05 2022] DELETE /identity/v3/projects/50c08341b4514b148a60922002cf3593 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:05.527112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52e7a-5e31-4183-b3fb-6ab976abafd6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:05.527432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52e7a-5e31-4183-b3fb-6ab976abafd6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:05.527713 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52e7a-5e31-4183-b3fb-6ab976abafd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:05.551920 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8df52e7a-5e31-4183-b3fb-6ab976abafd6 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:05.554602 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8df52e7a-5e31-4183-b3fb-6ab976abafd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:05.655061 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1369/2742] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:05.665712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7465-e422-4f9f-8496-7cb61363b96a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:05.666290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7465-e422-4f9f-8496-7cb61363b96a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:05.666471 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7465-e422-4f9f-8496-7cb61363b96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:05.689752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-579c7465-e422-4f9f-8496-7cb61363b96a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:05.691009 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-579c7465-e422-4f9f-8496-7cb61363b96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:05.699401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1374/2743] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:05.703714 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52e7a-5e31-4183-b3fb-6ab976abafd6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:05.714218 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52e7a-5e31-4183-b3fb-6ab976abafd6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:05.719322 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd716ff5-1071-440a-8fef-aebf36c454fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:05.720012 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd716ff5-1071-440a-8fef-aebf36c454fd admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:05.720303 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd716ff5-1071-440a-8fef-aebf36c454fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:05.720581 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd716ff5-1071-440a-8fef-aebf36c454fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:05.721469 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd716ff5-1071-440a-8fef-aebf36c454fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:23:05.721667 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd716ff5-1071-440a-8fef-aebf36c454fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:05.723292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd716ff5-1071-440a-8fef-aebf36c454fd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:05.737731 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1370/2744] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:23:05.743913 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c7465-e422-4f9f-8496-7cb61363b96a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:05.769054 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c7465-e422-4f9f-8496-7cb61363b96a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:05.773848 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d407f-3966-4261-aa6b-2b309d0571be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:05.774595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d407f-3966-4261-aa6b-2b309d0571be admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:05.774822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d407f-3966-4261-aa6b-2b309d0571be admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:05.775084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d407f-3966-4261-aa6b-2b309d0571be admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:05.775852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d407f-3966-4261-aa6b-2b309d0571be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:05.777790 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d407f-3966-4261-aa6b-2b309d0571be admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:05.806736 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1375/2745] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:05 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:05.811659 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd716ff5-1071-440a-8fef-aebf36c454fd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:05.822083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd716ff5-1071-440a-8fef-aebf36c454fd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:05.893779 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eeaa0-2d79-4a80-ab85-9fd0aee2c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:05.894857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eeaa0-2d79-4a80-ab85-9fd0aee2c47f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:05.894857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eeaa0-2d79-4a80-ab85-9fd0aee2c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:05.895222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eeaa0-2d79-4a80-ab85-9fd0aee2c47f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:05.895753 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0eeaa0-2d79-4a80-ab85-9fd0aee2c47f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:05.898079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0eeaa0-2d79-4a80-ab85-9fd0aee2c47f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:05.929915 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1371/2746] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:05 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:05.931174 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d407f-3966-4261-aa6b-2b309d0571be admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:05.950171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d407f-3966-4261-aa6b-2b309d0571be admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:05.958944 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a894f5-87b5-40d3-9349-02451194fda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:05.960879 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a894f5-87b5-40d3-9349-02451194fda7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:05.961290 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a894f5-87b5-40d3-9349-02451194fda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:05.961737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a894f5-87b5-40d3-9349-02451194fda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:05.963762 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a894f5-87b5-40d3-9349-02451194fda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:05.970418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a894f5-87b5-40d3-9349-02451194fda7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:05.987556 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1376/2747] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:23:05.991760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eeaa0-2d79-4a80-ab85-9fd0aee2c47f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:06.002738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eeaa0-2d79-4a80-ab85-9fd0aee2c47f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:06.008594 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49577d6-f5de-4562-b628-b0dccaee5abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:06.009332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49577d6-f5de-4562-b628-b0dccaee5abc admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.009560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49577d6-f5de-4562-b628-b0dccaee5abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.009896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49577d6-f5de-4562-b628-b0dccaee5abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.010632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49577d6-f5de-4562-b628-b0dccaee5abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:06.012479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49577d6-f5de-4562-b628-b0dccaee5abc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:06.024856 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1372/2748] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:06.029992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a894f5-87b5-40d3-9349-02451194fda7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:06.041362 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a894f5-87b5-40d3-9349-02451194fda7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:06.048416 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ee932-30ec-432d-920e-b5d493405b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:06.049527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ee932-30ec-432d-920e-b5d493405b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.049882 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ee932-30ec-432d-920e-b5d493405b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.050226 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ee932-30ec-432d-920e-b5d493405b34 admin admin] PATH_INFO: `/v3/projects/b997cd73e1104dd5be24a5a658706430/users/ded5a8ac513447dbbf0a4b8c63a6670b/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.064898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:06.076000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ee932-30ec-432d-920e-b5d493405b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b997cd73e1104dd5be24a5a658706430, user_id=ded5a8ac513447dbbf0a4b8c63a6670b, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:06.079950 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ee932-30ec-432d-920e-b5d493405b34 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:06.096622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1377/2749] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:06 2022] PUT /identity/v3/projects/b997cd73e1104dd5be24a5a658706430/users/ded5a8ac513447dbbf0a4b8c63a6670b/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:06.111292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0de23-0666-4c96-95b5-406b9aa284f2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.111922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0de23-0666-4c96-95b5-406b9aa284f2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.112651 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0de23-0666-4c96-95b5-406b9aa284f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.144074 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cdc0de23-0666-4c96-95b5-406b9aa284f2 None None] MFA Rules not processed for user `ded5a8ac513447dbbf0a4b8c63a6670b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:06.145799 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cdc0de23-0666-4c96-95b5-406b9aa284f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:06.221692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:06.236789 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1373/2750] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:06.242135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ee932-30ec-432d-920e-b5d493405b34 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:06.341700 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ee932-30ec-432d-920e-b5d493405b34 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:06.350325 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0a73c-1a8c-4d77-b4b4-51540ff37668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:06.351130 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0a73c-1a8c-4d77-b4b4-51540ff37668 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.351562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0a73c-1a8c-4d77-b4b4-51540ff37668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.352088 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0a73c-1a8c-4d77-b4b4-51540ff37668 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.353186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0a73c-1a8c-4d77-b4b4-51540ff37668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:06.356562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0a73c-1a8c-4d77-b4b4-51540ff37668 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:06.403822 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1378/2751] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:06 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:06.409717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0de23-0666-4c96-95b5-406b9aa284f2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:06.428497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0de23-0666-4c96-95b5-406b9aa284f2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:06.515388 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c15fb-9310-4d24-898f-58f164351f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:06.516085 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c15fb-9310-4d24-898f-58f164351f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.516369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c15fb-9310-4d24-898f-58f164351f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.516657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c15fb-9310-4d24-898f-58f164351f18 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.517434 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c15fb-9310-4d24-898f-58f164351f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:06.520097 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c15fb-9310-4d24-898f-58f164351f18 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:06.546335 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1374/2752] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:06 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:06.550625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0a73c-1a8c-4d77-b4b4-51540ff37668 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:06.563613 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0a73c-1a8c-4d77-b4b4-51540ff37668 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:06.570480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d5123-7fb2-4902-b0ac-15384985301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:06.571176 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d5123-7fb2-4902-b0ac-15384985301d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.571558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d5123-7fb2-4902-b0ac-15384985301d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.572197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d5123-7fb2-4902-b0ac-15384985301d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.573036 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d5123-7fb2-4902-b0ac-15384985301d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:06.575359 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d5123-7fb2-4902-b0ac-15384985301d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:06.589313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1379/2753] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:06.594412 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c15fb-9310-4d24-898f-58f164351f18 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:06.616945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c15fb-9310-4d24-898f-58f164351f18 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:06.630498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c8d6e-5ccf-423f-821b-345da9492268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:06.633376 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c8d6e-5ccf-423f-821b-345da9492268 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.633984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c8d6e-5ccf-423f-821b-345da9492268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.636297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c8d6e-5ccf-423f-821b-345da9492268 admin admin] PATH_INFO: `/v3/projects/c61c29fd0f0a4ea6bae7f3c745261203/users/7d20cf823c344881bd094a50e468cddd/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.657167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c8d6e-5ccf-423f-821b-345da9492268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61c29fd0f0a4ea6bae7f3c745261203, user_id=7d20cf823c344881bd094a50e468cddd, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:06.660045 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c8d6e-5ccf-423f-821b-345da9492268 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:06.682607 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1375/2754] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:06 2022] PUT /identity/v3/projects/c61c29fd0f0a4ea6bae7f3c745261203/users/7d20cf823c344881bd094a50e468cddd/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:06.688109 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d5123-7fb2-4902-b0ac-15384985301d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:06.706811 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d5123-7fb2-4902-b0ac-15384985301d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:06.797392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b4303-9560-4dc0-8504-fbea92daebea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:06.798266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b4303-9560-4dc0-8504-fbea92daebea admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.798560 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b4303-9560-4dc0-8504-fbea92daebea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.798844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b4303-9560-4dc0-8504-fbea92daebea admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.799636 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b4303-9560-4dc0-8504-fbea92daebea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:06.801353 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b4303-9560-4dc0-8504-fbea92daebea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:06.819321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1380/2755] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:23:06.820656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c8d6e-5ccf-423f-821b-345da9492268 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:06.834956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c8d6e-5ccf-423f-821b-345da9492268 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:06.840975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c80d3-299f-4f66-975b-6ed280e97412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:06.842430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c80d3-299f-4f66-975b-6ed280e97412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.842794 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c80d3-299f-4f66-975b-6ed280e97412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.843087 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c80d3-299f-4f66-975b-6ed280e97412 admin admin] PATH_INFO: `/v3/projects/c61c29fd0f0a4ea6bae7f3c745261203/users/7d20cf823c344881bd094a50e468cddd/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.846130 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c80d3-299f-4f66-975b-6ed280e97412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61c29fd0f0a4ea6bae7f3c745261203, user_id=7d20cf823c344881bd094a50e468cddd, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:06.849013 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c80d3-299f-4f66-975b-6ed280e97412 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:06.863983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1376/2756] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:06 2022] PUT /identity/v3/projects/c61c29fd0f0a4ea6bae7f3c745261203/users/7d20cf823c344881bd094a50e468cddd/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:06.875759 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057aed6a-e802-44b0-bebc-78e4d26995cd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:06.876172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057aed6a-e802-44b0-bebc-78e4d26995cd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:06.876172 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057aed6a-e802-44b0-bebc-78e4d26995cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:06.916546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-057aed6a-e802-44b0-bebc-78e4d26995cd None None] MFA Rules not processed for user `7d20cf823c344881bd094a50e468cddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:06.917725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-057aed6a-e802-44b0-bebc-78e4d26995cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:07.006908 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:07.042262 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1381/2757] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:23:07.056956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c80d3-299f-4f66-975b-6ed280e97412 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:07.059158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-280c80d3-299f-4f66-975b-6ed280e97412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:07.166801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c80d3-299f-4f66-975b-6ed280e97412 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:07.173560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431445f-89a7-4870-864e-7307a16a4958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:07.184777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431445f-89a7-4870-864e-7307a16a4958 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:07.184777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431445f-89a7-4870-864e-7307a16a4958 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:07.184777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431445f-89a7-4870-864e-7307a16a4958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:07.186315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431445f-89a7-4870-864e-7307a16a4958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:07.187827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431445f-89a7-4870-864e-7307a16a4958 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:07.200958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1377/2758] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:23:07.230994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057aed6a-e802-44b0-bebc-78e4d26995cd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:07.311904 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057aed6a-e802-44b0-bebc-78e4d26995cd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:07.317085 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:07.317998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:07.318288 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:07.318561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] PATH_INFO: `/v3/users/ded5a8ac513447dbbf0a4b8c63a6670b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:07.320544 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded5a8ac513447dbbf0a4b8c63a6670b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:07.322340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:07.411454 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded5a8ac513447dbbf0a4b8c63a6670b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:07.424032 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded5a8ac513447dbbf0a4b8c63a6670b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:07.436872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded5a8ac513447dbbf0a4b8c63a6670b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:07.453956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded5a8ac513447dbbf0a4b8c63a6670b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:07.454835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded5a8ac513447dbbf0a4b8c63a6670b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:07.464360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1382/2759] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:07 2022] DELETE /identity/v3/users/ded5a8ac513447dbbf0a4b8c63a6670b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:07.470727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c3eae-effb-4d7a-b890-1786a37f3ac8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:07.471421 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c3eae-effb-4d7a-b890-1786a37f3ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:07.471955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c3eae-effb-4d7a-b890-1786a37f3ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:07.503503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d77c3eae-effb-4d7a-b890-1786a37f3ac8 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:07.505503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d77c3eae-effb-4d7a-b890-1786a37f3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:07.587852 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:07.619758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1378/2760] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:23:07.629169 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:07.631148 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:07.729483 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39dbfb-c388-44fe-b246-3092719d2eac admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:07.737068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf927c-c827-44ae-9cd5-5e5cdbb4085a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:07.737640 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf927c-c827-44ae-9cd5-5e5cdbb4085a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:07.737981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf927c-c827-44ae-9cd5-5e5cdbb4085a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:07.738284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf927c-c827-44ae-9cd5-5e5cdbb4085a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:07.752635 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf927c-c827-44ae-9cd5-5e5cdbb4085a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:07.754572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf927c-c827-44ae-9cd5-5e5cdbb4085a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:07.770616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1383/2761] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:08.172348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c3eae-effb-4d7a-b890-1786a37f3ac8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:08.174139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d77c3eae-effb-4d7a-b890-1786a37f3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:08.198884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c3eae-effb-4d7a-b890-1786a37f3ac8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:08.205316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:08.206403 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:08.206701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:08.206985 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] PATH_INFO: `/v3/projects/b997cd73e1104dd5be24a5a658706430` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:08.208320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b997cd73e1104dd5be24a5a658706430)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:08.210999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:08.292147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b997cd73e1104dd5be24a5a658706430', 'request_id': 'req-645f7cd8-6161-4715-90cf-c4ace9d0c679'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.301050 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b997cd73e1104dd5be24a5a658706430', 'request_id': 'req-645f7cd8-6161-4715-90cf-c4ace9d0c679'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.309545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b997cd73e1104dd5be24a5a658706430', 'request_id': 'req-645f7cd8-6161-4715-90cf-c4ace9d0c679'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.350803 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] The token cache is being invalidate because project b997cd73e1104dd5be24a5a658706430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:08.351144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.352057 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.354222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1379/2762] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:08 2022] DELETE /identity/v3/projects/b997cd73e1104dd5be24a5a658706430 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:08.362876 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf927c-c827-44ae-9cd5-5e5cdbb4085a service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:08.382650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf927c-c827-44ae-9cd5-5e5cdbb4085a service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:08.388143 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:08.480999 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:08.482599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:08.484619 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:08.485466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] PATH_INFO: `/v3/users/7d20cf823c344881bd094a50e468cddd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:08.489074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d20cf823c344881bd094a50e468cddd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:08.491493 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:08.582017 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d20cf823c344881bd094a50e468cddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.596817 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d20cf823c344881bd094a50e468cddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.610857 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d20cf823c344881bd094a50e468cddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.638426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d20cf823c344881bd094a50e468cddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.639489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2d1aa34-00cf-4055-8197-655d2f1a345f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d20cf823c344881bd094a50e468cddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:08.648314 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1384/2763] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:08 2022] DELETE /identity/v3/users/7d20cf823c344881bd094a50e468cddd => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:09.052096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:09.054687 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:09.083162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f7cd8-6161-4715-90cf-c4ace9d0c679 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:09.164519 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:09.165395 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.165878 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.166165 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] PATH_INFO: `/v3/projects/c61c29fd0f0a4ea6bae7f3c745261203` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.167455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61c29fd0f0a4ea6bae7f3c745261203)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:09.169850 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:09.259914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61c29fd0f0a4ea6bae7f3c745261203', 'request_id': 'req-a1083de1-a540-46e3-8a47-6262a6b15827'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:09.268293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61c29fd0f0a4ea6bae7f3c745261203', 'request_id': 'req-a1083de1-a540-46e3-8a47-6262a6b15827'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:09.275291 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61c29fd0f0a4ea6bae7f3c745261203', 'request_id': 'req-a1083de1-a540-46e3-8a47-6262a6b15827'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:09.291381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] The token cache is being invalidate because project c61c29fd0f0a4ea6bae7f3c745261203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:09.291838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:09.292505 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1083de1-a540-46e3-8a47-6262a6b15827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:09.294168 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1380/2764] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:09 2022] DELETE /identity/v3/projects/c61c29fd0f0a4ea6bae7f3c745261203 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:09.299828 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22df7d-c284-4b68-bfb1-ae3093ecc22f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.300344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22df7d-c284-4b68-bfb1-ae3093ecc22f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.300442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22df7d-c284-4b68-bfb1-ae3093ecc22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.323568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9c22df7d-c284-4b68-bfb1-ae3093ecc22f None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:09.325726 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9c22df7d-c284-4b68-bfb1-ae3093ecc22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:09.434963 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1385/2765] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:23:09.448374 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbc59c-9479-4720-850e-a0369aefd959 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.448697 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbc59c-9479-4720-850e-a0369aefd959 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.448939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbc59c-9479-4720-850e-a0369aefd959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.458645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:09.478185 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-dedbc59c-9479-4720-850e-a0369aefd959 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:09.479866 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dedbc59c-9479-4720-850e-a0369aefd959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:09.493391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1381/2766] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:09.497860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22df7d-c284-4b68-bfb1-ae3093ecc22f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:09.510375 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22df7d-c284-4b68-bfb1-ae3093ecc22f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:09.516867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149e243-5e50-41dd-971f-db90ceb7748d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:09.517691 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149e243-5e50-41dd-971f-db90ceb7748d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.518073 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149e243-5e50-41dd-971f-db90ceb7748d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.518429 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149e243-5e50-41dd-971f-db90ceb7748d admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.519107 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149e243-5e50-41dd-971f-db90ceb7748d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:23:09.519452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149e243-5e50-41dd-971f-db90ceb7748d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:09.521151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149e243-5e50-41dd-971f-db90ceb7748d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:09.536460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1386/2767] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:23:09.542548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbc59c-9479-4720-850e-a0369aefd959 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:09.560051 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbc59c-9479-4720-850e-a0369aefd959 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:09.568153 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1696123-0e0e-42c6-8c88-a55171e436a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:09.569167 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1696123-0e0e-42c6-8c88-a55171e436a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.569703 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1696123-0e0e-42c6-8c88-a55171e436a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.570064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1696123-0e0e-42c6-8c88-a55171e436a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.571346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1696123-0e0e-42c6-8c88-a55171e436a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:09.574381 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1696123-0e0e-42c6-8c88-a55171e436a8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:09.613546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1382/2768] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:09 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:09.620307 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149e243-5e50-41dd-971f-db90ceb7748d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:09.634016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149e243-5e50-41dd-971f-db90ceb7748d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:09.650014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:09.734994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550330af-bc5a-404a-9965-cad34c1671e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:09.736076 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550330af-bc5a-404a-9965-cad34c1671e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.736594 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550330af-bc5a-404a-9965-cad34c1671e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.736971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550330af-bc5a-404a-9965-cad34c1671e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.738224 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550330af-bc5a-404a-9965-cad34c1671e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:09.740782 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550330af-bc5a-404a-9965-cad34c1671e2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:09.774768 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1387/2769] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:09 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:09.776413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1696123-0e0e-42c6-8c88-a55171e436a8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:09.788804 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1696123-0e0e-42c6-8c88-a55171e436a8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:09.795559 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df388656-4134-4075-ad1e-f4a83129c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:09.796430 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df388656-4134-4075-ad1e-f4a83129c402 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.796930 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df388656-4134-4075-ad1e-f4a83129c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.798081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df388656-4134-4075-ad1e-f4a83129c402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.799062 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df388656-4134-4075-ad1e-f4a83129c402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:09.800952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df388656-4134-4075-ad1e-f4a83129c402 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:09.815501 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1383/2770] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:23:09.819890 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550330af-bc5a-404a-9965-cad34c1671e2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:09.833052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550330af-bc5a-404a-9965-cad34c1671e2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:09.841418 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5466e-1394-40d8-9815-8fd9d567343e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:09.843583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5466e-1394-40d8-9815-8fd9d567343e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.843994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5466e-1394-40d8-9815-8fd9d567343e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.844527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5466e-1394-40d8-9815-8fd9d567343e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.846184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5466e-1394-40d8-9815-8fd9d567343e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:09.850749 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5466e-1394-40d8-9815-8fd9d567343e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:09.874729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1388/2771] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:09.877880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df388656-4134-4075-ad1e-f4a83129c402 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:09.891371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df388656-4134-4075-ad1e-f4a83129c402 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:09.896623 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c69d12-8956-460a-b7ee-254d86b343fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:09.897765 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c69d12-8956-460a-b7ee-254d86b343fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.898107 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c69d12-8956-460a-b7ee-254d86b343fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.898320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c69d12-8956-460a-b7ee-254d86b343fa admin admin] PATH_INFO: `/v3/projects/5df9fe50533c448996ae581694d03244/users/63b3a94016e145ce8153e9c9afd41fc7/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.923351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c69d12-8956-460a-b7ee-254d86b343fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df9fe50533c448996ae581694d03244, user_id=63b3a94016e145ce8153e9c9afd41fc7, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:09.928642 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c69d12-8956-460a-b7ee-254d86b343fa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:09.950027 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1384/2772] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:09 2022] PUT /identity/v3/projects/5df9fe50533c448996ae581694d03244/users/63b3a94016e145ce8153e9c9afd41fc7/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:09.961815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890546d-b5c8-4278-bf27-547d5cf130ec None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:09.962616 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890546d-b5c8-4278-bf27-547d5cf130ec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:09.963029 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890546d-b5c8-4278-bf27-547d5cf130ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:09.997531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7890546d-b5c8-4278-bf27-547d5cf130ec None None] MFA Rules not processed for user `63b3a94016e145ce8153e9c9afd41fc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:09.998764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7890546d-b5c8-4278-bf27-547d5cf130ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:10.084033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1389/2773] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:10.090598 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c69d12-8956-460a-b7ee-254d86b343fa admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:10.201662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c69d12-8956-460a-b7ee-254d86b343fa admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:10.208004 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40be00-fed3-4ca8-92a6-fe4dd501ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:10.208922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40be00-fed3-4ca8-92a6-fe4dd501ea15 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:10.209415 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40be00-fed3-4ca8-92a6-fe4dd501ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:10.209762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40be00-fed3-4ca8-92a6-fe4dd501ea15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:10.210963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40be00-fed3-4ca8-92a6-fe4dd501ea15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:10.213834 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40be00-fed3-4ca8-92a6-fe4dd501ea15 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:10.255763 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1385/2774] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:10 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:10.257119 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890546d-b5c8-4278-bf27-547d5cf130ec None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:10.277127 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890546d-b5c8-4278-bf27-547d5cf130ec None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:10.362583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b0445-174c-47fc-a99c-e9a4165e5719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:10.363683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b0445-174c-47fc-a99c-e9a4165e5719 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:10.363846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b0445-174c-47fc-a99c-e9a4165e5719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:10.363992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b0445-174c-47fc-a99c-e9a4165e5719 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:10.364932 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b0445-174c-47fc-a99c-e9a4165e5719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:10.367442 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b0445-174c-47fc-a99c-e9a4165e5719 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:10.396463 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1390/2775] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:10 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:10.402967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40be00-fed3-4ca8-92a6-fe4dd501ea15 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:10.422154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40be00-fed3-4ca8-92a6-fe4dd501ea15 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:10.427721 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d806a-d626-435d-a6c3-4ff0412ff6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:10.428424 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d806a-d626-435d-a6c3-4ff0412ff6bb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:10.429141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d806a-d626-435d-a6c3-4ff0412ff6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:10.429141 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d806a-d626-435d-a6c3-4ff0412ff6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:10.429869 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d806a-d626-435d-a6c3-4ff0412ff6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:10.431545 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d806a-d626-435d-a6c3-4ff0412ff6bb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:10.445336 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1386/2776] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:23:10.450902 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b0445-174c-47fc-a99c-e9a4165e5719 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:10.463212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b0445-174c-47fc-a99c-e9a4165e5719 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:10.470135 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f515-08e2-40f6-8a78-f73e74b0965e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:10.471285 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f515-08e2-40f6-8a78-f73e74b0965e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:10.471587 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f515-08e2-40f6-8a78-f73e74b0965e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:10.471868 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f515-08e2-40f6-8a78-f73e74b0965e admin admin] PATH_INFO: `/v3/projects/4c70ba48d5e747a499b9fc05ac999ec1/users/af3a62321bb94bfc9830f6a788ea1752/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:10.481863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:10.492259 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37f515-08e2-40f6-8a78-f73e74b0965e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c70ba48d5e747a499b9fc05ac999ec1, user_id=af3a62321bb94bfc9830f6a788ea1752, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:10.496110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37f515-08e2-40f6-8a78-f73e74b0965e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:10.517102 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1391/2777] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:10 2022] PUT /identity/v3/projects/4c70ba48d5e747a499b9fc05ac999ec1/users/af3a62321bb94bfc9830f6a788ea1752/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:10.521442 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d806a-d626-435d-a6c3-4ff0412ff6bb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:10.533138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d806a-d626-435d-a6c3-4ff0412ff6bb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:10.618822 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea3c1b-e7db-453b-b9e9-75a64afe048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:10.619702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea3c1b-e7db-453b-b9e9-75a64afe048d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:10.620005 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea3c1b-e7db-453b-b9e9-75a64afe048d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:10.620351 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea3c1b-e7db-453b-b9e9-75a64afe048d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:10.621113 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea3c1b-e7db-453b-b9e9-75a64afe048d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:10.623530 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea3c1b-e7db-453b-b9e9-75a64afe048d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:10.643312 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1387/2778] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:23:10.645713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f515-08e2-40f6-8a78-f73e74b0965e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:10.665010 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f515-08e2-40f6-8a78-f73e74b0965e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:10.674150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb579365-04ba-4ee3-bd24-271d09515839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:10.675923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb579365-04ba-4ee3-bd24-271d09515839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:10.677016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb579365-04ba-4ee3-bd24-271d09515839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:10.677662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb579365-04ba-4ee3-bd24-271d09515839 admin admin] PATH_INFO: `/v3/projects/4c70ba48d5e747a499b9fc05ac999ec1/users/af3a62321bb94bfc9830f6a788ea1752/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:10.681574 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb579365-04ba-4ee3-bd24-271d09515839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c70ba48d5e747a499b9fc05ac999ec1, user_id=af3a62321bb94bfc9830f6a788ea1752, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:10.687559 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb579365-04ba-4ee3-bd24-271d09515839 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:10.723356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1392/2779] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:10 2022] PUT /identity/v3/projects/4c70ba48d5e747a499b9fc05ac999ec1/users/af3a62321bb94bfc9830f6a788ea1752/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:10.733748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbe199-e446-49ea-9111-7a52d6cb0575 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:10.734173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbe199-e446-49ea-9111-7a52d6cb0575 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:10.734518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbe199-e446-49ea-9111-7a52d6cb0575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:10.770372 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:10.785798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-28fbe199-e446-49ea-9111-7a52d6cb0575 None None] MFA Rules not processed for user `af3a62321bb94bfc9830f6a788ea1752`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:10.787075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-28fbe199-e446-49ea-9111-7a52d6cb0575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:10.908344 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1388/2780] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:23:10.924521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb579365-04ba-4ee3-bd24-271d09515839 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:10.927206 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fb579365-04ba-4ee3-bd24-271d09515839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:11.016896 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb579365-04ba-4ee3-bd24-271d09515839 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:11.021770 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3222da0-72dc-4636-8500-20e0e4e5ec20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:11.022298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3222da0-72dc-4636-8500-20e0e4e5ec20 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:11.022586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3222da0-72dc-4636-8500-20e0e4e5ec20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:11.022860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3222da0-72dc-4636-8500-20e0e4e5ec20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:11.034356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3222da0-72dc-4636-8500-20e0e4e5ec20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:11.036402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3222da0-72dc-4636-8500-20e0e4e5ec20 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:11.050217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1393/2781] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:11.173273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fbe199-e446-49ea-9111-7a52d6cb0575 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:11.183751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fbe199-e446-49ea-9111-7a52d6cb0575 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:11.188560 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9f4c3-8e49-4766-8eb8-2b0fda20f740 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:11.189487 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9f4c3-8e49-4766-8eb8-2b0fda20f740 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:11.189709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9f4c3-8e49-4766-8eb8-2b0fda20f740 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:11.189967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9f4c3-8e49-4766-8eb8-2b0fda20f740 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:11.281417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9f4c3-8e49-4766-8eb8-2b0fda20f740 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:11.284261 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9f4c3-8e49-4766-8eb8-2b0fda20f740 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:11.298539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1389/2782] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:16.861678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3222da0-72dc-4636-8500-20e0e4e5ec20 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:16.979688 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3222da0-72dc-4636-8500-20e0e4e5ec20 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:16.988027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:16.989578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:16.989877 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:16.990170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] PATH_INFO: `/v3/users/63b3a94016e145ce8153e9c9afd41fc7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:16.991725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b3a94016e145ce8153e9c9afd41fc7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:16.993846 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:17.069761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b3a94016e145ce8153e9c9afd41fc7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:17.082564 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b3a94016e145ce8153e9c9afd41fc7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:17.095558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b3a94016e145ce8153e9c9afd41fc7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:17.111596 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:17.113903 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b3a94016e145ce8153e9c9afd41fc7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:17.114994 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b3a94016e145ce8153e9c9afd41fc7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:17.127855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1394/2783] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:16 2022] DELETE /identity/v3/users/63b3a94016e145ce8153e9c9afd41fc7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:17.132471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f7c23-e35a-48d5-b8f3-ddfa7c48a9c1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:17.133390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f7c23-e35a-48d5-b8f3-ddfa7c48a9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:17.133390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f7c23-e35a-48d5-b8f3-ddfa7c48a9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:17.151688 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8b4f7c23-e35a-48d5-b8f3-ddfa7c48a9c1 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:17.152965 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8b4f7c23-e35a-48d5-b8f3-ddfa7c48a9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:17.245584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1390/2784] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:17.256380 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:17.264383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:17.371391 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4494c-a72b-4d52-8b22-70402863e976 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:17.376757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d127d8-0073-495a-92c0-fef60a2de13b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:17.377426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d127d8-0073-495a-92c0-fef60a2de13b service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:17.377706 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d127d8-0073-495a-92c0-fef60a2de13b service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:17.378071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d127d8-0073-495a-92c0-fef60a2de13b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:17.395098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d127d8-0073-495a-92c0-fef60a2de13b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:17.397283 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d127d8-0073-495a-92c0-fef60a2de13b service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:17.414985 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1395/2785] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:17.849188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f7c23-e35a-48d5-b8f3-ddfa7c48a9c1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:17.852771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8b4f7c23-e35a-48d5-b8f3-ddfa7c48a9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:17.893882 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f7c23-e35a-48d5-b8f3-ddfa7c48a9c1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:17.910841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:17.912196 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:17.912698 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:17.913230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] PATH_INFO: `/v3/projects/5df9fe50533c448996ae581694d03244` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:17.915286 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df9fe50533c448996ae581694d03244)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:17.919657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:17.974137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:17.995081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df9fe50533c448996ae581694d03244', 'request_id': 'req-a85f7005-0d3f-4ae3-a72a-84fe35878b55'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.003155 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df9fe50533c448996ae581694d03244', 'request_id': 'req-a85f7005-0d3f-4ae3-a72a-84fe35878b55'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.012360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df9fe50533c448996ae581694d03244', 'request_id': 'req-a85f7005-0d3f-4ae3-a72a-84fe35878b55'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.033685 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] The token cache is being invalidate because project 5df9fe50533c448996ae581694d03244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:18.034285 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.035334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.037158 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1391/2786] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:17 2022] DELETE /identity/v3/projects/5df9fe50533c448996ae581694d03244 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:18.041751 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d127d8-0073-495a-92c0-fef60a2de13b service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:18.055668 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d127d8-0073-495a-92c0-fef60a2de13b service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:18.151992 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:18.153008 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:18.153355 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:18.153757 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] PATH_INFO: `/v3/users/af3a62321bb94bfc9830f6a788ea1752` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:18.155514 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3a62321bb94bfc9830f6a788ea1752)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:18.157728 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:18.243289 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3a62321bb94bfc9830f6a788ea1752'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.266600 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3a62321bb94bfc9830f6a788ea1752'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.281099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3a62321bb94bfc9830f6a788ea1752'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.313330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3a62321bb94bfc9830f6a788ea1752'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.314495 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b63d16bc-71f5-4992-9296-3437dc95bdea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3a62321bb94bfc9830f6a788ea1752'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:18.329042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1396/2787] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:18 2022] DELETE /identity/v3/users/af3a62321bb94bfc9830f6a788ea1752 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:18.790954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:18.793693 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:18.811471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7005-0d3f-4ae3-a72a-84fe35878b55 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:18.909275 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:18.910181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:18.910473 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:18.910748 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] PATH_INFO: `/v3/projects/4c70ba48d5e747a499b9fc05ac999ec1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:18.920320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c70ba48d5e747a499b9fc05ac999ec1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:18.923443 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:19.020795 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c70ba48d5e747a499b9fc05ac999ec1', 'request_id': 'req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:19.039009 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c70ba48d5e747a499b9fc05ac999ec1', 'request_id': 'req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:19.046922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c70ba48d5e747a499b9fc05ac999ec1', 'request_id': 'req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:19.069177 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] The token cache is being invalidate because project 4c70ba48d5e747a499b9fc05ac999ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:19.069778 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:19.070457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cc856f0-a4f0-41ea-8532-7f221d7dbf26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:19.072463 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1392/2788] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:18 2022] DELETE /identity/v3/projects/4c70ba48d5e747a499b9fc05ac999ec1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:19.078155 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cb6f2-2afe-4b85-be22-099fca761f24 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.079205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cb6f2-2afe-4b85-be22-099fca761f24 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.079205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cb6f2-2afe-4b85-be22-099fca761f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:19.106147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c57cb6f2-2afe-4b85-be22-099fca761f24 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:19.107687 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c57cb6f2-2afe-4b85-be22-099fca761f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:19.124159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:19.206573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1397/2789] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:19.214618 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a3ff-7863-4cb9-b253-024553e933e4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.214928 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a3ff-7863-4cb9-b253-024553e933e4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.215090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a3ff-7863-4cb9-b253-024553e933e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:19.245662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7616a3ff-7863-4cb9-b253-024553e933e4 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:19.246955 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7616a3ff-7863-4cb9-b253-024553e933e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:19.261527 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cb6f2-2afe-4b85-be22-099fca761f24 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:19.267077 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1393/2790] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:19.276132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cb6f2-2afe-4b85-be22-099fca761f24 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:19.287042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea7dbd-232c-4740-a506-bc5116eb32e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:19.288370 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea7dbd-232c-4740-a506-bc5116eb32e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.288810 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea7dbd-232c-4740-a506-bc5116eb32e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.289199 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea7dbd-232c-4740-a506-bc5116eb32e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:19.290617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea7dbd-232c-4740-a506-bc5116eb32e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:23:19.291162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea7dbd-232c-4740-a506-bc5116eb32e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:19.294875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea7dbd-232c-4740-a506-bc5116eb32e5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:19.310578 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1398/2791] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:23:19.315064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a3ff-7863-4cb9-b253-024553e933e4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:19.327504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a3ff-7863-4cb9-b253-024553e933e4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:19.334603 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e2d5c-d6f1-491b-843f-167be42780d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:19.335650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e2d5c-d6f1-491b-843f-167be42780d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.336073 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e2d5c-d6f1-491b-843f-167be42780d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.336543 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e2d5c-d6f1-491b-843f-167be42780d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:19.337758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e2d5c-d6f1-491b-843f-167be42780d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:19.340568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e2d5c-d6f1-491b-843f-167be42780d7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:19.381101 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1394/2792] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:19 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:19.387363 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea7dbd-232c-4740-a506-bc5116eb32e5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:19.406114 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea7dbd-232c-4740-a506-bc5116eb32e5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:19.512535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f190b-efaf-43ec-8966-e313eb219724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:19.514139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f190b-efaf-43ec-8966-e313eb219724 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.514670 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f190b-efaf-43ec-8966-e313eb219724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.515187 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f190b-efaf-43ec-8966-e313eb219724 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:19.516621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f190b-efaf-43ec-8966-e313eb219724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:19.520284 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f190b-efaf-43ec-8966-e313eb219724 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:19.561548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1399/2793] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:19 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:23:19.562191 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e2d5c-d6f1-491b-843f-167be42780d7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:19.572873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e2d5c-d6f1-491b-843f-167be42780d7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:19.579017 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18181a-263d-43a8-9728-4e602692fb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:19.579814 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18181a-263d-43a8-9728-4e602692fb6c admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.580103 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18181a-263d-43a8-9728-4e602692fb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.580385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18181a-263d-43a8-9728-4e602692fb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:19.581156 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18181a-263d-43a8-9728-4e602692fb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:19.582768 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18181a-263d-43a8-9728-4e602692fb6c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:19.596689 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1395/2794] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:19.602223 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f190b-efaf-43ec-8966-e313eb219724 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:19.618080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f190b-efaf-43ec-8966-e313eb219724 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:19.624360 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398020f-894e-43dd-abfe-9856fc22ed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:19.625120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398020f-894e-43dd-abfe-9856fc22ed7e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.625452 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398020f-894e-43dd-abfe-9856fc22ed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.625733 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398020f-894e-43dd-abfe-9856fc22ed7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:19.626477 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398020f-894e-43dd-abfe-9856fc22ed7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:19.628294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398020f-894e-43dd-abfe-9856fc22ed7e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:19.647409 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1400/2795] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:19.651042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18181a-263d-43a8-9728-4e602692fb6c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:19.675180 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18181a-263d-43a8-9728-4e602692fb6c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:19.683788 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abfaa9-c3fd-4242-81b1-2edff6e24aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:19.684696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abfaa9-c3fd-4242-81b1-2edff6e24aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.685683 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abfaa9-c3fd-4242-81b1-2edff6e24aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.686164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abfaa9-c3fd-4242-81b1-2edff6e24aa8 admin admin] PATH_INFO: `/v3/projects/766b35ef85b14a22b8cf4ea1585bd180/users/a76c73d3582247f69cf4abebae132f18/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:19.691235 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:19.714048 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abfaa9-c3fd-4242-81b1-2edff6e24aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=766b35ef85b14a22b8cf4ea1585bd180, user_id=a76c73d3582247f69cf4abebae132f18, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:19.717139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abfaa9-c3fd-4242-81b1-2edff6e24aa8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:19.741049 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1396/2796] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:19 2022] PUT /identity/v3/projects/766b35ef85b14a22b8cf4ea1585bd180/users/a76c73d3582247f69cf4abebae132f18/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:19.746482 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734cc83-6bd0-4774-a4a6-bd0224e9e7cd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.746918 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734cc83-6bd0-4774-a4a6-bd0224e9e7cd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.747508 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734cc83-6bd0-4774-a4a6-bd0224e9e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:19.773777 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5734cc83-6bd0-4774-a4a6-bd0224e9e7cd None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:19.775294 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5734cc83-6bd0-4774-a4a6-bd0224e9e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:19.879572 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1401/2797] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:19.890070 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abfaa9-c3fd-4242-81b1-2edff6e24aa8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:19.892362 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-60abfaa9-c3fd-4242-81b1-2edff6e24aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:19.989542 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abfaa9-c3fd-4242-81b1-2edff6e24aa8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:19.997701 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc9e3-b30f-4aae-939f-6e80c0163404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:19.999054 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc9e3-b30f-4aae-939f-6e80c0163404 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:19.999585 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc9e3-b30f-4aae-939f-6e80c0163404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:19.999880 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc9e3-b30f-4aae-939f-6e80c0163404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:20.012134 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfc9e3-b30f-4aae-939f-6e80c0163404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:20.013635 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfc9e3-b30f-4aae-939f-6e80c0163404 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:20.025871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1397/2798] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:20.944816 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d4dee-f99a-4afb-8889-21d81a9a80f0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:20.945396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d4dee-f99a-4afb-8889-21d81a9a80f0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:20.945396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d4dee-f99a-4afb-8889-21d81a9a80f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:20.995342 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d81d4dee-f99a-4afb-8889-21d81a9a80f0 None None] MFA Rules not processed for user `a76c73d3582247f69cf4abebae132f18`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:20.997420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d81d4dee-f99a-4afb-8889-21d81a9a80f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:21.023250 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:21.125305 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1402/2799] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:21.132491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc9e3-b30f-4aae-939f-6e80c0163404 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:21.142827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc9e3-b30f-4aae-939f-6e80c0163404 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:21.154628 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0f89f-9e0d-4b3f-bda1-a4926f48eb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:21.157782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0f89f-9e0d-4b3f-bda1-a4926f48eb5c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:21.157782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0f89f-9e0d-4b3f-bda1-a4926f48eb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:21.157782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0f89f-9e0d-4b3f-bda1-a4926f48eb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:21.157782 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0f89f-9e0d-4b3f-bda1-a4926f48eb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:21.161371 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0f89f-9e0d-4b3f-bda1-a4926f48eb5c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:21.222616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1398/2800] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:21 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:21.228996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d4dee-f99a-4afb-8889-21d81a9a80f0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:21.253595 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d4dee-f99a-4afb-8889-21d81a9a80f0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:21.373488 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692a76-7b98-47f9-9b3b-b6e141b2ede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:21.375122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692a76-7b98-47f9-9b3b-b6e141b2ede7 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:21.376006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692a76-7b98-47f9-9b3b-b6e141b2ede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:21.376729 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692a76-7b98-47f9-9b3b-b6e141b2ede7 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:21.378252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88692a76-7b98-47f9-9b3b-b6e141b2ede7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:21.382601 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88692a76-7b98-47f9-9b3b-b6e141b2ede7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:21.412614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1403/2801] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:21 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:23:21.418401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0f89f-9e0d-4b3f-bda1-a4926f48eb5c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:21.436040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0f89f-9e0d-4b3f-bda1-a4926f48eb5c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:21.442289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186c619-bb04-4432-b894-e07d66e7e7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:21.443298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186c619-bb04-4432-b894-e07d66e7e7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:21.443737 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186c619-bb04-4432-b894-e07d66e7e7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:21.444108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186c619-bb04-4432-b894-e07d66e7e7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:21.445064 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186c619-bb04-4432-b894-e07d66e7e7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:21.447037 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186c619-bb04-4432-b894-e07d66e7e7cb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:21.465398 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1399/2802] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:21.470540 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692a76-7b98-47f9-9b3b-b6e141b2ede7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:21.487843 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692a76-7b98-47f9-9b3b-b6e141b2ede7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:21.493902 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b19035-f1fa-4633-8d15-5b5fb892bf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:21.494713 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b19035-f1fa-4633-8d15-5b5fb892bf50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:21.495195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b19035-f1fa-4633-8d15-5b5fb892bf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:21.495340 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b19035-f1fa-4633-8d15-5b5fb892bf50 admin admin] PATH_INFO: `/v3/projects/acd340f4066e4e50a67608ac06380c20/users/149ead2f6e9b4cd18783c2d55540449c/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:21.519195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b19035-f1fa-4633-8d15-5b5fb892bf50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acd340f4066e4e50a67608ac06380c20, user_id=149ead2f6e9b4cd18783c2d55540449c, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:21.522998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b19035-f1fa-4633-8d15-5b5fb892bf50 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:21.542996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1404/2803] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:21 2022] PUT /identity/v3/projects/acd340f4066e4e50a67608ac06380c20/users/149ead2f6e9b4cd18783c2d55540449c/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:21.550280 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186c619-bb04-4432-b894-e07d66e7e7cb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:21.564392 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186c619-bb04-4432-b894-e07d66e7e7cb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:21.681164 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fd230-d347-4dba-a9cf-7262cead6887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:21.681884 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd230-d347-4dba-a9cf-7262cead6887 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:21.682175 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd230-d347-4dba-a9cf-7262cead6887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:21.682458 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd230-d347-4dba-a9cf-7262cead6887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:21.683203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fd230-d347-4dba-a9cf-7262cead6887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:21.684875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fd230-d347-4dba-a9cf-7262cead6887 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:21.700095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1400/2804] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:21.705321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b19035-f1fa-4633-8d15-5b5fb892bf50 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:21.719474 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b19035-f1fa-4633-8d15-5b5fb892bf50 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:21.726815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46b4ac-5922-4f25-96a5-45b6b8990f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:21.727754 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46b4ac-5922-4f25-96a5-45b6b8990f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:21.728042 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46b4ac-5922-4f25-96a5-45b6b8990f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:21.728317 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46b4ac-5922-4f25-96a5-45b6b8990f89 admin admin] PATH_INFO: `/v3/projects/acd340f4066e4e50a67608ac06380c20/users/149ead2f6e9b4cd18783c2d55540449c/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:21.730308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46b4ac-5922-4f25-96a5-45b6b8990f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acd340f4066e4e50a67608ac06380c20, user_id=149ead2f6e9b4cd18783c2d55540449c, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:21.733044 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46b4ac-5922-4f25-96a5-45b6b8990f89 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:21.743996 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1405/2805] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:21 2022] PUT /identity/v3/projects/acd340f4066e4e50a67608ac06380c20/users/149ead2f6e9b4cd18783c2d55540449c/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:22.856367 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b662c0-49a1-4ce0-84d0-32c3be1b4d27 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:22.857316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b662c0-49a1-4ce0-84d0-32c3be1b4d27 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:22.857316 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b662c0-49a1-4ce0-84d0-32c3be1b4d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:22.917297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c3b662c0-49a1-4ce0-84d0-32c3be1b4d27 None None] MFA Rules not processed for user `149ead2f6e9b4cd18783c2d55540449c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:22.920377 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c3b662c0-49a1-4ce0-84d0-32c3be1b4d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:23.005188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:23.050041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1401/2806] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:23.085065 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46b4ac-5922-4f25-96a5-45b6b8990f89 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:23.086936 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4e46b4ac-5922-4f25-96a5-45b6b8990f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:23.227046 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46b4ac-5922-4f25-96a5-45b6b8990f89 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:23.231339 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48cf63-90de-46ad-b4ab-7117e407270d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:23.232118 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48cf63-90de-46ad-b4ab-7117e407270d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:23.232426 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48cf63-90de-46ad-b4ab-7117e407270d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:23.232705 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48cf63-90de-46ad-b4ab-7117e407270d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:23.267962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:23.330888 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48cf63-90de-46ad-b4ab-7117e407270d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:23.334518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48cf63-90de-46ad-b4ab-7117e407270d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:23.354779 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1406/2807] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:24.989791 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b662c0-49a1-4ce0-84d0-32c3be1b4d27 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:25.000332 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b662c0-49a1-4ce0-84d0-32c3be1b4d27 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:25.004509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cd35e-9b0a-427e-afbe-74f49480f844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:25.005162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cd35e-9b0a-427e-afbe-74f49480f844 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:25.005452 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cd35e-9b0a-427e-afbe-74f49480f844 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:25.005975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cd35e-9b0a-427e-afbe-74f49480f844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:25.016116 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cd35e-9b0a-427e-afbe-74f49480f844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:25.017653 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cd35e-9b0a-427e-afbe-74f49480f844 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:25.029457 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1402/2808] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:25.768010 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5d2e8-04ef-404f-bd49-e398f63c9134 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:25.768536 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5d2e8-04ef-404f-bd49-e398f63c9134 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:25.768684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5d2e8-04ef-404f-bd49-e398f63c9134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:25.787789 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5fb5d2e8-04ef-404f-bd49-e398f63c9134 None None] MFA Rules not processed for user `646d7c5e6c8e4825a7fe06d018ee319c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:25.789037 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5fb5d2e8-04ef-404f-bd49-e398f63c9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:25.846004 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1407/2809] 10.222.0.20 () {58 vars in 967 bytes} [Fri Jul 22 11:23:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:23:25.858873 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cd35e-9b0a-427e-afbe-74f49480f844 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:25.861118 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-605cd35e-9b0a-427e-afbe-74f49480f844 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:25.935993 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cd35e-9b0a-427e-afbe-74f49480f844 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:25.940847 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb481fc3-9f93-4ed0-ad09-aaa357e4c972 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7935e85cc3f4ed1ad5b01ba892e6e82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:25.941509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb481fc3-9f93-4ed0-ad09-aaa357e4c972 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:25.941796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb481fc3-9f93-4ed0-ad09-aaa357e4c972 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:25.942099 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb481fc3-9f93-4ed0-ad09-aaa357e4c972 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:25.952808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb481fc3-9f93-4ed0-ad09-aaa357e4c972 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:25.953283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb481fc3-9f93-4ed0-ad09-aaa357e4c972 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:25.963390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1403/2810] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Jul 22 11:23:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 22 11:23:28.010485 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5d2e8-04ef-404f-bd49-e398f63c9134 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:28.012026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5fb5d2e8-04ef-404f-bd49-e398f63c9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:28.096537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5d2e8-04ef-404f-bd49-e398f63c9134 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:28.100981 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:28.101867 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.102150 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.102424 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] PATH_INFO: `/v3/users/2e46f6bf93bf4e839a0dfebd5409f32e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.103847 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e46f6bf93bf4e839a0dfebd5409f32e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:28.105842 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:28.173346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e46f6bf93bf4e839a0dfebd5409f32e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.191895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e46f6bf93bf4e839a0dfebd5409f32e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.199679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e46f6bf93bf4e839a0dfebd5409f32e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.204151 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:28.213844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e46f6bf93bf4e839a0dfebd5409f32e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.214741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a3e0f19-7f6e-49e7-8cf9-970cd65ea6d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e46f6bf93bf4e839a0dfebd5409f32e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.221249 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1408/2811] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:28 2022] DELETE /identity/v3/users/2e46f6bf93bf4e839a0dfebd5409f32e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:28.401429 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb481fc3-9f93-4ed0-ad09-aaa357e4c972 service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:28.413470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb481fc3-9f93-4ed0-ad09-aaa357e4c972 service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:28.475350 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:28.476225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.476512 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.476790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] PATH_INFO: `/v3/projects/17c28a2aaa4b4d4bada90cea2b8ccff5` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.478066 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c28a2aaa4b4d4bada90cea2b8ccff5)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:28.480465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:28.527401 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c28a2aaa4b4d4bada90cea2b8ccff5', 'request_id': 'req-e159154c-491a-4468-a723-cea0b652e57a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.532466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:28.534010 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c28a2aaa4b4d4bada90cea2b8ccff5', 'request_id': 'req-e159154c-491a-4468-a723-cea0b652e57a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.542468 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c28a2aaa4b4d4bada90cea2b8ccff5', 'request_id': 'req-e159154c-491a-4468-a723-cea0b652e57a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.556176 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] The token cache is being invalidate because project 17c28a2aaa4b4d4bada90cea2b8ccff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:28.556459 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.557081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e159154c-491a-4468-a723-cea0b652e57a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:28.558671 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1404/2812] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:28 2022] DELETE /identity/v3/projects/17c28a2aaa4b4d4bada90cea2b8ccff5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:28.563893 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adbae4-66f7-44e3-83b0-75d1e066db51 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.564188 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adbae4-66f7-44e3-83b0-75d1e066db51 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.564469 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adbae4-66f7-44e3-83b0-75d1e066db51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.583866 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b0adbae4-66f7-44e3-83b0-75d1e066db51 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:28.585103 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b0adbae4-66f7-44e3-83b0-75d1e066db51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:28.649662 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1409/2813] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:28.658031 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b9aff-af67-4617-8835-713b00301b03 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.658321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b9aff-af67-4617-8835-713b00301b03 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.658631 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b9aff-af67-4617-8835-713b00301b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.677836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d00b9aff-af67-4617-8835-713b00301b03 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:28.679058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d00b9aff-af67-4617-8835-713b00301b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:28.687342 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1405/2814] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:28.691808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0adbae4-66f7-44e3-83b0-75d1e066db51 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:28.701519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0adbae4-66f7-44e3-83b0-75d1e066db51 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:28.706068 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660ed02-c87d-4835-bd17-c138bd1f702e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:28.706720 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660ed02-c87d-4835-bd17-c138bd1f702e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.707001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660ed02-c87d-4835-bd17-c138bd1f702e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.707299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660ed02-c87d-4835-bd17-c138bd1f702e admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.708026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660ed02-c87d-4835-bd17-c138bd1f702e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:23:28.708329 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660ed02-c87d-4835-bd17-c138bd1f702e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:28.709918 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660ed02-c87d-4835-bd17-c138bd1f702e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:28.722747 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1410/2815] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:23:28.727247 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b9aff-af67-4617-8835-713b00301b03 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:28.737278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b9aff-af67-4617-8835-713b00301b03 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:28.742260 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbe54b-e6b3-4f0d-bb38-8d32ed360016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:28.742916 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbe54b-e6b3-4f0d-bb38-8d32ed360016 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.743207 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbe54b-e6b3-4f0d-bb38-8d32ed360016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.743522 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbe54b-e6b3-4f0d-bb38-8d32ed360016 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.744299 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbe54b-e6b3-4f0d-bb38-8d32ed360016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:28.746216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbe54b-e6b3-4f0d-bb38-8d32ed360016 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:28.771811 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1406/2816] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:28 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:28.776035 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660ed02-c87d-4835-bd17-c138bd1f702e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:28.791648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660ed02-c87d-4835-bd17-c138bd1f702e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:28.850265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ef0ea-a8f8-4c66-9bf2-4f921c2bf315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:28.850943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ef0ea-a8f8-4c66-9bf2-4f921c2bf315 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.851238 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ef0ea-a8f8-4c66-9bf2-4f921c2bf315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.851547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ef0ea-a8f8-4c66-9bf2-4f921c2bf315 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.852243 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ef0ea-a8f8-4c66-9bf2-4f921c2bf315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:28.854165 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ef0ea-a8f8-4c66-9bf2-4f921c2bf315 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:28.874139 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1411/2817] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:28 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:28.878269 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbe54b-e6b3-4f0d-bb38-8d32ed360016 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:28.889694 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbe54b-e6b3-4f0d-bb38-8d32ed360016 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:28.896041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d620a-e9fb-4736-9177-8a1d11f47ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:28.896696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d620a-e9fb-4736-9177-8a1d11f47ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.896980 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d620a-e9fb-4736-9177-8a1d11f47ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.897257 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d620a-e9fb-4736-9177-8a1d11f47ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.897986 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d620a-e9fb-4736-9177-8a1d11f47ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:28.899598 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d620a-e9fb-4736-9177-8a1d11f47ba2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:28.910936 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1407/2818] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:28.914898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ef0ea-a8f8-4c66-9bf2-4f921c2bf315 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:28.925197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ef0ea-a8f8-4c66-9bf2-4f921c2bf315 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:28.929874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897994b-2a2a-4b83-8e98-34d3346f6945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:28.930537 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897994b-2a2a-4b83-8e98-34d3346f6945 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.930841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897994b-2a2a-4b83-8e98-34d3346f6945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.931096 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897994b-2a2a-4b83-8e98-34d3346f6945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.931851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897994b-2a2a-4b83-8e98-34d3346f6945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:28.933561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897994b-2a2a-4b83-8e98-34d3346f6945 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:28.943683 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1412/2819] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:28.948453 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d620a-e9fb-4736-9177-8a1d11f47ba2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:28.959510 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d620a-e9fb-4736-9177-8a1d11f47ba2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:28.965263 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98ca65-cebd-43e8-9ac8-47d570bf38dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:28.965438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98ca65-cebd-43e8-9ac8-47d570bf38dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:28.965678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98ca65-cebd-43e8-9ac8-47d570bf38dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:28.966862 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98ca65-cebd-43e8-9ac8-47d570bf38dd admin admin] PATH_INFO: `/v3/projects/d084bf56f7894fb98a46d7e2cbb34157/users/11af2fb64a6c4a4ea660196aa90a9c9e/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:28.982509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98ca65-cebd-43e8-9ac8-47d570bf38dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d084bf56f7894fb98a46d7e2cbb34157, user_id=11af2fb64a6c4a4ea660196aa90a9c9e, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:28.985199 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98ca65-cebd-43e8-9ac8-47d570bf38dd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:28.999684 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1408/2820] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:28 2022] PUT /identity/v3/projects/d084bf56f7894fb98a46d7e2cbb34157/users/11af2fb64a6c4a4ea660196aa90a9c9e/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:29.004614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33775c63-de56-4e2c-9bba-1f5abc4851cb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:29.004902 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33775c63-de56-4e2c-9bba-1f5abc4851cb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:29.005195 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33775c63-de56-4e2c-9bba-1f5abc4851cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:29.024190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-33775c63-de56-4e2c-9bba-1f5abc4851cb None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:29.025489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-33775c63-de56-4e2c-9bba-1f5abc4851cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:29.090835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:29.109147 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1413/2821] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:29.119161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98ca65-cebd-43e8-9ac8-47d570bf38dd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:29.121404 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4b98ca65-cebd-43e8-9ac8-47d570bf38dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:29.183568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98ca65-cebd-43e8-9ac8-47d570bf38dd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:29.190605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cd2a2-f694-4330-8dee-bad2faada216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:29.191281 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cd2a2-f694-4330-8dee-bad2faada216 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:29.191576 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cd2a2-f694-4330-8dee-bad2faada216 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:29.191857 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cd2a2-f694-4330-8dee-bad2faada216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:29.202756 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37cd2a2-f694-4330-8dee-bad2faada216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:29.204232 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37cd2a2-f694-4330-8dee-bad2faada216 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:29.217277 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1409/2822] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:34.119486 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcf80e-76be-4f2f-91f2-bd0535bc052a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:34.119974 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcf80e-76be-4f2f-91f2-bd0535bc052a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:34.120074 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcf80e-76be-4f2f-91f2-bd0535bc052a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:34.148605 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-85fcf80e-76be-4f2f-91f2-bd0535bc052a None None] MFA Rules not processed for user `11af2fb64a6c4a4ea660196aa90a9c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:34.149986 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-85fcf80e-76be-4f2f-91f2-bd0535bc052a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:34.227925 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1414/2823] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:34.232823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cd2a2-f694-4330-8dee-bad2faada216 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:34.243621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cd2a2-f694-4330-8dee-bad2faada216 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:34.249264 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a811e-171b-4c8a-9d99-3503d9a7d317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:34.249961 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a811e-171b-4c8a-9d99-3503d9a7d317 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:34.250246 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a811e-171b-4c8a-9d99-3503d9a7d317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:34.250520 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a811e-171b-4c8a-9d99-3503d9a7d317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:34.251223 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a811e-171b-4c8a-9d99-3503d9a7d317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:34.253226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a811e-171b-4c8a-9d99-3503d9a7d317 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:34.286154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1410/2824] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:34 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:34.291614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcf80e-76be-4f2f-91f2-bd0535bc052a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:34.307614 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcf80e-76be-4f2f-91f2-bd0535bc052a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:34.373949 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5d512-4948-4779-8147-fce10bedd237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:34.374673 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5d512-4948-4779-8147-fce10bedd237 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:34.374998 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5d512-4948-4779-8147-fce10bedd237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:34.375310 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5d512-4948-4779-8147-fce10bedd237 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:34.376018 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5d512-4948-4779-8147-fce10bedd237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:34.378298 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5d512-4948-4779-8147-fce10bedd237 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:34.399274 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1415/2825] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:34 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:34.403967 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a811e-171b-4c8a-9d99-3503d9a7d317 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:34.415293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a811e-171b-4c8a-9d99-3503d9a7d317 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:34.420622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255ae55-d370-4dc0-9b40-3713d2760fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:34.421331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255ae55-d370-4dc0-9b40-3713d2760fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:34.421622 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255ae55-d370-4dc0-9b40-3713d2760fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:34.421909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255ae55-d370-4dc0-9b40-3713d2760fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:34.422657 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255ae55-d370-4dc0-9b40-3713d2760fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:34.424315 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255ae55-d370-4dc0-9b40-3713d2760fe9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:34.435696 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1411/2826] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:34.439914 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5d512-4948-4779-8147-fce10bedd237 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:34.450571 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5d512-4948-4779-8147-fce10bedd237 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:34.455414 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dcd62-6edc-4d00-9d44-d51565255301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:34.456321 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dcd62-6edc-4d00-9d44-d51565255301 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:34.456611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dcd62-6edc-4d00-9d44-d51565255301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:34.456895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dcd62-6edc-4d00-9d44-d51565255301 admin admin] PATH_INFO: `/v3/projects/efb23709e5fc4d6ea06087c45b13f0e9/users/b460166917df4ee49cb59f942fed5a51/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:34.473878 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977dcd62-6edc-4d00-9d44-d51565255301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb23709e5fc4d6ea06087c45b13f0e9, user_id=b460166917df4ee49cb59f942fed5a51, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:34.476438 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977dcd62-6edc-4d00-9d44-d51565255301 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:34.492481 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1416/2827] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:34 2022] PUT /identity/v3/projects/efb23709e5fc4d6ea06087c45b13f0e9/users/b460166917df4ee49cb59f942fed5a51/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:34.496413 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255ae55-d370-4dc0-9b40-3713d2760fe9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:34.507112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255ae55-d370-4dc0-9b40-3713d2760fe9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:34.554803 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:34.569708 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a75657-f0b1-4061-aef9-73c30f69b0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:34.570417 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a75657-f0b1-4061-aef9-73c30f69b0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:34.570709 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a75657-f0b1-4061-aef9-73c30f69b0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:34.570997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a75657-f0b1-4061-aef9-73c30f69b0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:34.571770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a75657-f0b1-4061-aef9-73c30f69b0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:34.573479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a75657-f0b1-4061-aef9-73c30f69b0a5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:34.584225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1412/2828] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:34.588523 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dcd62-6edc-4d00-9d44-d51565255301 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:34.599912 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dcd62-6edc-4d00-9d44-d51565255301 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:34.605642 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db362f9-d898-42df-8979-9bb432970567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:34.607158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db362f9-d898-42df-8979-9bb432970567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:34.607515 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db362f9-d898-42df-8979-9bb432970567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:34.607864 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db362f9-d898-42df-8979-9bb432970567 admin admin] PATH_INFO: `/v3/projects/efb23709e5fc4d6ea06087c45b13f0e9/users/b460166917df4ee49cb59f942fed5a51/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:34.611231 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db362f9-d898-42df-8979-9bb432970567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb23709e5fc4d6ea06087c45b13f0e9, user_id=b460166917df4ee49cb59f942fed5a51, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:34.614629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db362f9-d898-42df-8979-9bb432970567 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:34.628690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1417/2829] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:34 2022] PUT /identity/v3/projects/efb23709e5fc4d6ea06087c45b13f0e9/users/b460166917df4ee49cb59f942fed5a51/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:39.874669 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41a5e1-0217-4327-ba89-b9a1b0f65999 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:39.875583 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41a5e1-0217-4327-ba89-b9a1b0f65999 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:39.876033 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41a5e1-0217-4327-ba89-b9a1b0f65999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:39.961516 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ff41a5e1-0217-4327-ba89-b9a1b0f65999 None None] MFA Rules not processed for user `b460166917df4ee49cb59f942fed5a51`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:39.962825 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ff41a5e1-0217-4327-ba89-b9a1b0f65999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:40.125007 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1413/2830] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:40.142666 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db362f9-d898-42df-8979-9bb432970567 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:40.145097 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9db362f9-d898-42df-8979-9bb432970567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:40.198027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:40.255808 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db362f9-d898-42df-8979-9bb432970567 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:40.264711 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c0d1c-7879-42e2-a92b-cf2e424768fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:40.265500 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c0d1c-7879-42e2-a92b-cf2e424768fe service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:40.266381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c0d1c-7879-42e2-a92b-cf2e424768fe service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:40.266381 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c0d1c-7879-42e2-a92b-cf2e424768fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:40.344411 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c0d1c-7879-42e2-a92b-cf2e424768fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:40.346163 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c0d1c-7879-42e2-a92b-cf2e424768fe service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:40.356939 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1418/2831] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:40.752181 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41a5e1-0217-4327-ba89-b9a1b0f65999 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:40.753954 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ff41a5e1-0217-4327-ba89-b9a1b0f65999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:40.869321 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41a5e1-0217-4327-ba89-b9a1b0f65999 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:40.875883 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:40.876798 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:40.877084 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:40.877365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] PATH_INFO: `/v3/users/a76c73d3582247f69cf4abebae132f18` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:40.878611 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76c73d3582247f69cf4abebae132f18)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:40.880599 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:40.942963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76c73d3582247f69cf4abebae132f18'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:40.942963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76c73d3582247f69cf4abebae132f18'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:40.953810 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76c73d3582247f69cf4abebae132f18'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:40.971853 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76c73d3582247f69cf4abebae132f18'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:40.973230 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:40.985574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76c73d3582247f69cf4abebae132f18'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:40.992384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1414/2832] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:40 2022] DELETE /identity/v3/users/a76c73d3582247f69cf4abebae132f18 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:41.213625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c0d1c-7879-42e2-a92b-cf2e424768fe service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:41.230690 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c0d1c-7879-42e2-a92b-cf2e424768fe service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:41.342910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:41.344055 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:41.344513 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:41.344895 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] PATH_INFO: `/v3/projects/766b35ef85b14a22b8cf4ea1585bd180` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:41.346547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=766b35ef85b14a22b8cf4ea1585bd180)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:41.348650 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:41.424480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '766b35ef85b14a22b8cf4ea1585bd180', 'request_id': 'req-65733fc7-e944-4c54-bbc9-663a09a14eba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.434794 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '766b35ef85b14a22b8cf4ea1585bd180', 'request_id': 'req-65733fc7-e944-4c54-bbc9-663a09a14eba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.441558 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '766b35ef85b14a22b8cf4ea1585bd180', 'request_id': 'req-65733fc7-e944-4c54-bbc9-663a09a14eba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.453823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:41.460761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] The token cache is being invalidate because project 766b35ef85b14a22b8cf4ea1585bd180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:41.460761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.461929 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-65733fc7-e944-4c54-bbc9-663a09a14eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.464247 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1419/2833] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:41 2022] DELETE /identity/v3/projects/766b35ef85b14a22b8cf4ea1585bd180 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:41.468998 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:41.470550 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:41.485875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013a4d5-6da8-4158-9b74-dc5261aca2ad admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:41.550996 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:41.551912 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:41.552274 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:41.552650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] PATH_INFO: `/v3/users/149ead2f6e9b4cd18783c2d55540449c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:41.553914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=149ead2f6e9b4cd18783c2d55540449c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:41.555953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:41.615858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96327c-5b4a-4d65-a9fd-9e30fdb0caed None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:41.616159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96327c-5b4a-4d65-a9fd-9e30fdb0caed None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:41.616443 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96327c-5b4a-4d65-a9fd-9e30fdb0caed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:41.625747 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149ead2f6e9b4cd18783c2d55540449c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.626498 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149ead2f6e9b4cd18783c2d55540449c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.634356 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149ead2f6e9b4cd18783c2d55540449c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.634531 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ab96327c-5b4a-4d65-a9fd-9e30fdb0caed None None] MFA Rules not processed for user `11af2fb64a6c4a4ea660196aa90a9c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:41.635841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ab96327c-5b4a-4d65-a9fd-9e30fdb0caed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:41.648136 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149ead2f6e9b4cd18783c2d55540449c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.657582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149ead2f6e9b4cd18783c2d55540449c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:41.667095 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1415/2834] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:41 2022] DELETE /identity/v3/users/149ead2f6e9b4cd18783c2d55540449c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:41.737806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1420/2835] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:41.932358 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:41.939357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:42.103582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaad4d-29e5-4b1d-a61c-e7a1fc68f856 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:42.113194 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:42.116455 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.118407 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.119823 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] PATH_INFO: `/v3/projects/acd340f4066e4e50a67608ac06380c20` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.124489 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acd340f4066e4e50a67608ac06380c20)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:42.126665 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:42.141926 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:42.210945 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acd340f4066e4e50a67608ac06380c20', 'request_id': 'req-bf4e3e39-5831-4949-a83f-230a73fca761'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:42.222040 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acd340f4066e4e50a67608ac06380c20', 'request_id': 'req-bf4e3e39-5831-4949-a83f-230a73fca761'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:42.230621 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acd340f4066e4e50a67608ac06380c20', 'request_id': 'req-bf4e3e39-5831-4949-a83f-230a73fca761'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:42.254282 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] The token cache is being invalidate because project acd340f4066e4e50a67608ac06380c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:42.254757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:42.255867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-bf4e3e39-5831-4949-a83f-230a73fca761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:42.258311 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1416/2836] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:41 2022] DELETE /identity/v3/projects/acd340f4066e4e50a67608ac06380c20 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:42.264410 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7b02b-792b-4bab-a0be-2d3ca7d34cd4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.265154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7b02b-792b-4bab-a0be-2d3ca7d34cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.265154 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7b02b-792b-4bab-a0be-2d3ca7d34cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.284624 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cee7b02b-792b-4bab-a0be-2d3ca7d34cd4 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:42.285930 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cee7b02b-792b-4bab-a0be-2d3ca7d34cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:42.377521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1421/2837] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:23:42.385274 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035c110-8de1-43df-a577-0c221e129d55 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.385574 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035c110-8de1-43df-a577-0c221e129d55 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.385956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035c110-8de1-43df-a577-0c221e129d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.409203 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4035c110-8de1-43df-a577-0c221e129d55 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:42.410662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4035c110-8de1-43df-a577-0c221e129d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:42.420598 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1417/2838] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:42.425031 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7b02b-792b-4bab-a0be-2d3ca7d34cd4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:42.435821 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7b02b-792b-4bab-a0be-2d3ca7d34cd4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:42.442872 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb387b3-896a-46bc-9e4d-e6d63dc2edd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:42.444098 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb387b3-896a-46bc-9e4d-e6d63dc2edd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.444573 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb387b3-896a-46bc-9e4d-e6d63dc2edd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.445016 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb387b3-896a-46bc-9e4d-e6d63dc2edd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.445966 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb387b3-896a-46bc-9e4d-e6d63dc2edd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:23:42.446446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb387b3-896a-46bc-9e4d-e6d63dc2edd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:42.448301 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb387b3-896a-46bc-9e4d-e6d63dc2edd9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:42.467308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1422/2839] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:23:42.469471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035c110-8de1-43df-a577-0c221e129d55 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:42.480893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035c110-8de1-43df-a577-0c221e129d55 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:42.486929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0c078-bc1e-43fc-8054-2bfbbce90092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:42.487662 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0c078-bc1e-43fc-8054-2bfbbce90092 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.487953 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0c078-bc1e-43fc-8054-2bfbbce90092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.488240 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0c078-bc1e-43fc-8054-2bfbbce90092 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.488956 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0c078-bc1e-43fc-8054-2bfbbce90092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:42.491138 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0c078-bc1e-43fc-8054-2bfbbce90092 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:42.523337 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1418/2840] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:42 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:42.527204 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb387b3-896a-46bc-9e4d-e6d63dc2edd9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:42.538948 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb387b3-896a-46bc-9e4d-e6d63dc2edd9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:42.610303 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62ec33-cef1-4e95-b378-ba50eedf4785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:42.611159 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62ec33-cef1-4e95-b378-ba50eedf4785 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.611528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62ec33-cef1-4e95-b378-ba50eedf4785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.612190 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62ec33-cef1-4e95-b378-ba50eedf4785 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.612620 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62ec33-cef1-4e95-b378-ba50eedf4785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:42.614917 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62ec33-cef1-4e95-b378-ba50eedf4785 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:42.641761 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1423/2841] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:42 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:42.647287 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0c078-bc1e-43fc-8054-2bfbbce90092 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:42.664042 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0c078-bc1e-43fc-8054-2bfbbce90092 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:42.672617 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6876a1-6ab1-4b82-9439-05fd3617fcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:42.673409 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6876a1-6ab1-4b82-9439-05fd3617fcb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.673710 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6876a1-6ab1-4b82-9439-05fd3617fcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.673994 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6876a1-6ab1-4b82-9439-05fd3617fcb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.675003 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6876a1-6ab1-4b82-9439-05fd3617fcb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:42.677348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6876a1-6ab1-4b82-9439-05fd3617fcb0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:42.693390 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1419/2842] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:42.694812 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62ec33-cef1-4e95-b378-ba50eedf4785 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:42.706588 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62ec33-cef1-4e95-b378-ba50eedf4785 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:42.712835 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465a6e1-5809-4d6e-a63e-344ded2e0ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:42.713561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465a6e1-5809-4d6e-a63e-344ded2e0ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.713851 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465a6e1-5809-4d6e-a63e-344ded2e0ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.714132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465a6e1-5809-4d6e-a63e-344ded2e0ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.715960 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465a6e1-5809-4d6e-a63e-344ded2e0ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:42.717171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465a6e1-5809-4d6e-a63e-344ded2e0ce8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:42.728730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1424/2843] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:42.733119 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6876a1-6ab1-4b82-9439-05fd3617fcb0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:42.745539 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6876a1-6ab1-4b82-9439-05fd3617fcb0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:42.758108 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66991d5-0d50-4e84-b8c1-4e4d4ead2d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:42.758914 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66991d5-0d50-4e84-b8c1-4e4d4ead2d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.759503 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66991d5-0d50-4e84-b8c1-4e4d4ead2d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.759969 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66991d5-0d50-4e84-b8c1-4e4d4ead2d96 admin admin] PATH_INFO: `/v3/projects/6b13935aee6d49a6bbcb90724ec29052/users/79755980bcdc4e5c9414ebde6c881c86/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.787295 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66991d5-0d50-4e84-b8c1-4e4d4ead2d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b13935aee6d49a6bbcb90724ec29052, user_id=79755980bcdc4e5c9414ebde6c881c86, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:42.789818 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66991d5-0d50-4e84-b8c1-4e4d4ead2d96 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:42.810393 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1420/2844] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:42 2022] PUT /identity/v3/projects/6b13935aee6d49a6bbcb90724ec29052/users/79755980bcdc4e5c9414ebde6c881c86/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:42.822845 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ceeda-997a-439b-9606-6d3a135e9bba None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:42.823357 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ceeda-997a-439b-9606-6d3a135e9bba None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:42.823889 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ceeda-997a-439b-9606-6d3a135e9bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:42.845446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:42.861856 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-690ceeda-997a-439b-9606-6d3a135e9bba None None] MFA Rules not processed for user `79755980bcdc4e5c9414ebde6c881c86`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:42.861856 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-690ceeda-997a-439b-9606-6d3a135e9bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:42.996897 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1425/2845] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:42.997187 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66991d5-0d50-4e84-b8c1-4e4d4ead2d96 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:43.096813 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66991d5-0d50-4e84-b8c1-4e4d4ead2d96 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:43.106885 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0614c-459d-4662-aa72-bd2d5f8c817f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:43.108274 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0614c-459d-4662-aa72-bd2d5f8c817f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:43.108562 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0614c-459d-4662-aa72-bd2d5f8c817f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:43.108841 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0614c-459d-4662-aa72-bd2d5f8c817f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:43.109762 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0614c-459d-4662-aa72-bd2d5f8c817f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:43.112222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0614c-459d-4662-aa72-bd2d5f8c817f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:43.191975 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1421/2846] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:42 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:23:43.192550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ceeda-997a-439b-9606-6d3a135e9bba None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:43.212313 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ceeda-997a-439b-9606-6d3a135e9bba None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:43.307784 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08b430-7cdb-4f69-bbd8-a3d110201a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:43.310343 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08b430-7cdb-4f69-bbd8-a3d110201a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:43.310746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08b430-7cdb-4f69-bbd8-a3d110201a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:43.311197 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08b430-7cdb-4f69-bbd8-a3d110201a58 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:43.312219 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08b430-7cdb-4f69-bbd8-a3d110201a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:43.316460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08b430-7cdb-4f69-bbd8-a3d110201a58 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:43.352090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1426/2847] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:43 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 22 11:23:43.356088 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0614c-459d-4662-aa72-bd2d5f8c817f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:43.369751 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0614c-459d-4662-aa72-bd2d5f8c817f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:43.376786 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6832a-d9f7-4515-9185-d65d550b693e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:43.377474 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6832a-d9f7-4515-9185-d65d550b693e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:43.377760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6832a-d9f7-4515-9185-d65d550b693e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:43.378036 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6832a-d9f7-4515-9185-d65d550b693e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:43.378773 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6832a-d9f7-4515-9185-d65d550b693e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:43.380429 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6832a-d9f7-4515-9185-d65d550b693e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:43.392399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1422/2848] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:43.396625 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08b430-7cdb-4f69-bbd8-a3d110201a58 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:43.410568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08b430-7cdb-4f69-bbd8-a3d110201a58 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:43.416543 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b778f55-753c-42b9-a7e3-1d7e99fea90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:43.417460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b778f55-753c-42b9-a7e3-1d7e99fea90c admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:43.417752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b778f55-753c-42b9-a7e3-1d7e99fea90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:43.418033 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b778f55-753c-42b9-a7e3-1d7e99fea90c admin admin] PATH_INFO: `/v3/projects/265fd1d35b834b4aa7b05d58e5b81d55/users/e65708a3cfaf4612a5758c19ac244d22/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:43.437007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b778f55-753c-42b9-a7e3-1d7e99fea90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=265fd1d35b834b4aa7b05d58e5b81d55, user_id=e65708a3cfaf4612a5758c19ac244d22, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:43.439917 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b778f55-753c-42b9-a7e3-1d7e99fea90c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:43.462528 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1427/2849] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:43 2022] PUT /identity/v3/projects/265fd1d35b834b4aa7b05d58e5b81d55/users/e65708a3cfaf4612a5758c19ac244d22/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:43.463435 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6832a-d9f7-4515-9185-d65d550b693e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:43.475582 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6832a-d9f7-4515-9185-d65d550b693e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:43.483022 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:43.554886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5f355-7307-4550-ab03-ebaef2ba6f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:43.556297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f355-7307-4550-ab03-ebaef2ba6f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:43.556667 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f355-7307-4550-ab03-ebaef2ba6f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:43.556952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f355-7307-4550-ab03-ebaef2ba6f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:43.557702 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5f355-7307-4550-ab03-ebaef2ba6f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:43.560019 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5f355-7307-4550-ab03-ebaef2ba6f41 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:43.571465 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1423/2850] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:23:43.577638 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b778f55-753c-42b9-a7e3-1d7e99fea90c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:43.591519 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b778f55-753c-42b9-a7e3-1d7e99fea90c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:43.601383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade0dd5-a427-467d-9f8f-9ac74689489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:43.602460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade0dd5-a427-467d-9f8f-9ac74689489b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:43.602860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade0dd5-a427-467d-9f8f-9ac74689489b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:43.603450 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade0dd5-a427-467d-9f8f-9ac74689489b admin admin] PATH_INFO: `/v3/projects/265fd1d35b834b4aa7b05d58e5b81d55/users/e65708a3cfaf4612a5758c19ac244d22/roles/7f92cc1bdd9240b887d0f2aaf4055734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:43.606142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade0dd5-a427-467d-9f8f-9ac74689489b admin admin] RBAC: Authorizing `identity:create_grant(project_id=265fd1d35b834b4aa7b05d58e5b81d55, user_id=e65708a3cfaf4612a5758c19ac244d22, role_id=7f92cc1bdd9240b887d0f2aaf4055734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:43.609248 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade0dd5-a427-467d-9f8f-9ac74689489b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:43.624718 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1428/2851] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:43 2022] PUT /identity/v3/projects/265fd1d35b834b4aa7b05d58e5b81d55/users/e65708a3cfaf4612a5758c19ac244d22/roles/7f92cc1bdd9240b887d0f2aaf4055734 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:43.636858 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154fcc1-4e46-4303-bf8b-0502f27e634c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:43.637169 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154fcc1-4e46-4303-bf8b-0502f27e634c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:43.637557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154fcc1-4e46-4303-bf8b-0502f27e634c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:43.676566 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1154fcc1-4e46-4303-bf8b-0502f27e634c None None] MFA Rules not processed for user `e65708a3cfaf4612a5758c19ac244d22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:43.678296 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1154fcc1-4e46-4303-bf8b-0502f27e634c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:43.779320 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1424/2852] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:43.794185 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade0dd5-a427-467d-9f8f-9ac74689489b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:43.797509 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bade0dd5-a427-467d-9f8f-9ac74689489b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:43.874346 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade0dd5-a427-467d-9f8f-9ac74689489b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:43.879364 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09562f3d-802f-4da6-97f3-398b3b77464b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:43.880731 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09562f3d-802f-4da6-97f3-398b3b77464b service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:43.881090 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09562f3d-802f-4da6-97f3-398b3b77464b service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:43.881557 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09562f3d-802f-4da6-97f3-398b3b77464b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:43.900128 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:44.040878 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09562f3d-802f-4da6-97f3-398b3b77464b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:44.042401 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09562f3d-802f-4da6-97f3-398b3b77464b service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:44.054923 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1429/2853] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:23:49.529290 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154fcc1-4e46-4303-bf8b-0502f27e634c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:49.530717 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1154fcc1-4e46-4303-bf8b-0502f27e634c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:49.544645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154fcc1-4e46-4303-bf8b-0502f27e634c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:49.548477 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7152fee-26ec-47ae-8e78-ddcae4f1f967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:49.549144 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7152fee-26ec-47ae-8e78-ddcae4f1f967 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:49.549432 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7152fee-26ec-47ae-8e78-ddcae4f1f967 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:49.549742 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7152fee-26ec-47ae-8e78-ddcae4f1f967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:49.562041 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7152fee-26ec-47ae-8e78-ddcae4f1f967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:49.563922 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7152fee-26ec-47ae-8e78-ddcae4f1f967 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:49.575575 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1425/2854] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:23:54.049082 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=24 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:54.724470 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09562f3d-802f-4da6-97f3-398b3b77464b service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:54.823631 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09562f3d-802f-4da6-97f3-398b3b77464b service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:54.829629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:54.830602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:54.830965 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:54.831349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] PATH_INFO: `/v3/users/79755980bcdc4e5c9414ebde6c881c86` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:54.832630 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79755980bcdc4e5c9414ebde6c881c86)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:54.834680 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:54.907113 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79755980bcdc4e5c9414ebde6c881c86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:54.926447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79755980bcdc4e5c9414ebde6c881c86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:54.943871 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79755980bcdc4e5c9414ebde6c881c86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:54.964772 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79755980bcdc4e5c9414ebde6c881c86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:54.966014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79755980bcdc4e5c9414ebde6c881c86'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:54.977192 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1430/2855] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:54 2022] DELETE /identity/v3/users/79755980bcdc4e5c9414ebde6c881c86 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:54.983777 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171647d-4cd0-429e-8e78-578b4af73548 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:54.984871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171647d-4cd0-429e-8e78-578b4af73548 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:54.984871 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171647d-4cd0-429e-8e78-578b4af73548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:55.007860 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c171647d-4cd0-429e-8e78-578b4af73548 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:55.009228 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c171647d-4cd0-429e-8e78-578b4af73548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:55.088616 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1426/2856] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 22 11:23:55.102328 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:55.104723 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:55.183200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2481e-f405-4f7a-b8f8-f0848e097c45 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:55.187214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9315e-af85-4900-bd58-8419411cd473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:55.187934 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9315e-af85-4900-bd58-8419411cd473 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:55.188244 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9315e-af85-4900-bd58-8419411cd473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:55.188568 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9315e-af85-4900-bd58-8419411cd473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:55.200001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9315e-af85-4900-bd58-8419411cd473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:55.201489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9315e-af85-4900-bd58-8419411cd473 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:55.212252 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1431/2857] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:23:55.535083 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171647d-4cd0-429e-8e78-578b4af73548 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:55.536589 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c171647d-4cd0-429e-8e78-578b4af73548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:55.551828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171647d-4cd0-429e-8e78-578b4af73548 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:55.563471 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:55.564370 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:55.564668 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:55.564947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] PATH_INFO: `/v3/projects/6b13935aee6d49a6bbcb90724ec29052` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:55.566760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b13935aee6d49a6bbcb90724ec29052)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:55.569128 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:55.583691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:55.627189 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b13935aee6d49a6bbcb90724ec29052', 'request_id': 'req-21939d69-0c2b-4009-b973-94dc03068d56'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.633174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b13935aee6d49a6bbcb90724ec29052', 'request_id': 'req-21939d69-0c2b-4009-b973-94dc03068d56'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.640334 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b13935aee6d49a6bbcb90724ec29052', 'request_id': 'req-21939d69-0c2b-4009-b973-94dc03068d56'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.652958 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] The token cache is being invalidate because project 6b13935aee6d49a6bbcb90724ec29052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:55.653249 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.653952 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.655673 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1427/2858] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:55 2022] DELETE /identity/v3/projects/6b13935aee6d49a6bbcb90724ec29052 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:55.659875 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9315e-af85-4900-bd58-8419411cd473 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:55.670599 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9315e-af85-4900-bd58-8419411cd473 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:55.691170 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:55.733142 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:55.733956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:55.734241 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:55.734518 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] PATH_INFO: `/v3/users/e65708a3cfaf4612a5758c19ac244d22` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:55.735752 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65708a3cfaf4612a5758c19ac244d22)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:55.737685 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:55.790498 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65708a3cfaf4612a5758c19ac244d22'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.798898 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65708a3cfaf4612a5758c19ac244d22'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.806445 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65708a3cfaf4612a5758c19ac244d22'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.819233 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65708a3cfaf4612a5758c19ac244d22'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.820000 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fdd5f451-c953-4fbd-839d-f73beec9ff53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65708a3cfaf4612a5758c19ac244d22'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:55.825990 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1432/2859] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:23:55 2022] DELETE /identity/v3/users/e65708a3cfaf4612a5758c19ac244d22 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:23:56.170838 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:56.172667 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:56.188226 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21939d69-0c2b-4009-b973-94dc03068d56 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:56.254595 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:56.255514 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.255801 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.256081 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] PATH_INFO: `/v3/projects/265fd1d35b834b4aa7b05d58e5b81d55` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.257331 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=265fd1d35b834b4aa7b05d58e5b81d55)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:56.259349 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:56.310836 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265fd1d35b834b4aa7b05d58e5b81d55', 'request_id': 'req-fba474c6-9beb-4bac-8f9d-917152d74510'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:56.316364 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265fd1d35b834b4aa7b05d58e5b81d55', 'request_id': 'req-fba474c6-9beb-4bac-8f9d-917152d74510'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:56.323030 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265fd1d35b834b4aa7b05d58e5b81d55', 'request_id': 'req-fba474c6-9beb-4bac-8f9d-917152d74510'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:56.342504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] The token cache is being invalidate because project 265fd1d35b834b4aa7b05d58e5b81d55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:23:56.342906 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:56.343738 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-fba474c6-9beb-4bac-8f9d-917152d74510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:23:56.345384 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1428/2860] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:23:56 2022] DELETE /identity/v3/projects/265fd1d35b834b4aa7b05d58e5b81d55 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:56.351465 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621322c-4960-492b-b954-29f5e8758d5b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.352212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621322c-4960-492b-b954-29f5e8758d5b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.352212 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621322c-4960-492b-b954-29f5e8758d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.373709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e621322c-4960-492b-b954-29f5e8758d5b None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:56.375344 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e621322c-4960-492b-b954-29f5e8758d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:56.451829 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1433/2861] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:56.461632 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c58326-cb4e-4787-bbba-4de9469c2f2a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.462363 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c58326-cb4e-4787-bbba-4de9469c2f2a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.462485 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c58326-cb4e-4787-bbba-4de9469c2f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.483212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f2c58326-cb4e-4787-bbba-4de9469c2f2a None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:56.484491 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f2c58326-cb4e-4787-bbba-4de9469c2f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:56.493619 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1429/2862] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:56.498702 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621322c-4960-492b-b954-29f5e8758d5b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:56.509606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621322c-4960-492b-b954-29f5e8758d5b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:56.515071 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a6c38-523a-4f20-8411-0d462c3d17dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:56.515783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a6c38-523a-4f20-8411-0d462c3d17dd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.516217 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a6c38-523a-4f20-8411-0d462c3d17dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.516347 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a6c38-523a-4f20-8411-0d462c3d17dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.517106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a6c38-523a-4f20-8411-0d462c3d17dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 22 11:23:56.517412 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a6c38-523a-4f20-8411-0d462c3d17dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:56.519027 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a6c38-523a-4f20-8411-0d462c3d17dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:56.533678 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1434/2863] 10.222.0.20 () {58 vars in 1152 bytes} [Fri Jul 22 11:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 22 11:23:56.538006 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c58326-cb4e-4787-bbba-4de9469c2f2a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:56.547828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c58326-cb4e-4787-bbba-4de9469c2f2a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:56.552718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3acc-76c9-4bf3-b843-3afd0d8102e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:56.553389 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3acc-76c9-4bf3-b843-3afd0d8102e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.553672 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3acc-76c9-4bf3-b843-3afd0d8102e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.553947 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3acc-76c9-4bf3-b843-3afd0d8102e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.554691 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e3acc-76c9-4bf3-b843-3afd0d8102e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:56.556624 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e3acc-76c9-4bf3-b843-3afd0d8102e1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:56.581924 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:56.600378 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1430/2864] 10.222.0.20 () {60 vars in 1152 bytes} [Fri Jul 22 11:23:56 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:56.604725 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a6c38-523a-4f20-8411-0d462c3d17dd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:56.615659 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a6c38-523a-4f20-8411-0d462c3d17dd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:56.644814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:23:56.680941 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5da68-75a5-4b3a-adec-e61ab7682d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:56.681628 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5da68-75a5-4b3a-adec-e61ab7682d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.681919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5da68-75a5-4b3a-adec-e61ab7682d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.682203 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5da68-75a5-4b3a-adec-e61ab7682d3a admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.683030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5da68-75a5-4b3a-adec-e61ab7682d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:56.686024 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5da68-75a5-4b3a-adec-e61ab7682d3a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:56.712030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1435/2865] 10.222.0.20 () {60 vars in 1143 bytes} [Fri Jul 22 11:23:56 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 22 11:23:56.713150 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3acc-76c9-4bf3-b843-3afd0d8102e1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:56.724114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3acc-76c9-4bf3-b843-3afd0d8102e1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:56.729824 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c1678-5e14-4b0d-aa5e-c728b36e1880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:56.731039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c1678-5e14-4b0d-aa5e-c728b36e1880 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.731360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c1678-5e14-4b0d-aa5e-c728b36e1880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.731655 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c1678-5e14-4b0d-aa5e-c728b36e1880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.732517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c1678-5e14-4b0d-aa5e-c728b36e1880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:56.734075 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c1678-5e14-4b0d-aa5e-c728b36e1880 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:56.745757 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1431/2866] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 22 11:23:56.750489 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5da68-75a5-4b3a-adec-e61ab7682d3a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:56.761001 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5da68-75a5-4b3a-adec-e61ab7682d3a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:56.766034 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cec5f-13e1-415d-a155-66eb8c0faae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:56.766722 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cec5f-13e1-415d-a155-66eb8c0faae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.767014 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cec5f-13e1-415d-a155-66eb8c0faae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.767337 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cec5f-13e1-415d-a155-66eb8c0faae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.768080 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cec5f-13e1-415d-a155-66eb8c0faae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:56.769709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cec5f-13e1-415d-a155-66eb8c0faae1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:56.780964 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1436/2867] 10.222.0.20 () {58 vars in 1121 bytes} [Fri Jul 22 11:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 22 11:23:56.785273 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c1678-5e14-4b0d-aa5e-c728b36e1880 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:56.796497 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c1678-5e14-4b0d-aa5e-c728b36e1880 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:56.801211 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb14c2c-1824-45ae-9d38-db966f25e315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:56.802112 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb14c2c-1824-45ae-9d38-db966f25e315 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.802410 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb14c2c-1824-45ae-9d38-db966f25e315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.802699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb14c2c-1824-45ae-9d38-db966f25e315 admin admin] PATH_INFO: `/v3/projects/26b009afec5a4e9d84281e6c8cf49c87/users/56bd9d4e97ea4648bef6ffbb4d071d77/roles/8eb9d526616b44efa3dcd0128b945ef8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.820206 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb14c2c-1824-45ae-9d38-db966f25e315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b009afec5a4e9d84281e6c8cf49c87, user_id=56bd9d4e97ea4648bef6ffbb4d071d77, role_id=8eb9d526616b44efa3dcd0128b945ef8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:56.822886 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb14c2c-1824-45ae-9d38-db966f25e315 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:56.836796 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1432/2868] 10.222.0.20 () {60 vars in 1482 bytes} [Fri Jul 22 11:23:56 2022] PUT /identity/v3/projects/26b009afec5a4e9d84281e6c8cf49c87/users/56bd9d4e97ea4648bef6ffbb4d071d77/roles/8eb9d526616b44efa3dcd0128b945ef8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:23:56.849308 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8cb58-812c-46e1-92bb-bb29e334dca3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:56.849606 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8cb58-812c-46e1-92bb-bb29e334dca3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:56.849910 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8cb58-812c-46e1-92bb-bb29e334dca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:56.879430 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-88f8cb58-812c-46e1-92bb-bb29e334dca3 None None] MFA Rules not processed for user `56bd9d4e97ea4648bef6ffbb4d071d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:23:56.880593 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-88f8cb58-812c-46e1-92bb-bb29e334dca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:56.947275 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1437/2869] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:23:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:23:56.958147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb14c2c-1824-45ae-9d38-db966f25e315 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:23:56.960021 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ebb14c2c-1824-45ae-9d38-db966f25e315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:23:57.021292 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb14c2c-1824-45ae-9d38-db966f25e315 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:23:57.024988 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8dee-852f-4c00-9040-f429a17ff820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:23:57.025650 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8dee-852f-4c00-9040-f429a17ff820 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:23:57.025941 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8dee-852f-4c00-9040-f429a17ff820 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:23:57.026225 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8dee-852f-4c00-9040-f429a17ff820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:23:57.035896 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d8dee-852f-4c00-9040-f429a17ff820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:23:57.037347 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d8dee-852f-4c00-9040-f429a17ff820 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:23:57.046991 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1433/2870] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:23:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:24:06.944178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:24:07.042216 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:24:07.834183 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed367cc-95e2-4713-99cf-9c2213643efa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:07.834480 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed367cc-95e2-4713-99cf-9c2213643efa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:07.834764 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed367cc-95e2-4713-99cf-9c2213643efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:07.852586 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3ed367cc-95e2-4713-99cf-9c2213643efa None None] MFA Rules not processed for user `00ab892a9233421886b849b879bd8a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:24:07.853741 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ed367cc-95e2-4713-99cf-9c2213643efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:07.912265 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1438/2871] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:24:17.905548 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=25 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:24:18.762815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8dee-852f-4c00-9040-f429a17ff820 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:24:18.829293 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8dee-852f-4c00-9040-f429a17ff820 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:24:18.833770 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:24:18.834640 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:18.834983 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:18.835266 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] PATH_INFO: `/v3/users/56bd9d4e97ea4648bef6ffbb4d071d77` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:18.836546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56bd9d4e97ea4648bef6ffbb4d071d77)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:24:18.838999 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:24:18.892357 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56bd9d4e97ea4648bef6ffbb4d071d77'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:18.893251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56bd9d4e97ea4648bef6ffbb4d071d77'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:18.902744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56bd9d4e97ea4648bef6ffbb4d071d77'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:18.924490 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56bd9d4e97ea4648bef6ffbb4d071d77'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:18.937039 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56bd9d4e97ea4648bef6ffbb4d071d77'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:18.944663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1434/2872] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:24:18 2022] DELETE /identity/v3/users/56bd9d4e97ea4648bef6ffbb4d071d77 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:24:18.949225 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aa5d6-d97d-41ed-854b-5704bb7f1811 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:18.949742 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aa5d6-d97d-41ed-854b-5704bb7f1811 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:18.949863 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aa5d6-d97d-41ed-854b-5704bb7f1811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:18.968806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-308aa5d6-d97d-41ed-854b-5704bb7f1811 None None] MFA Rules not processed for user `9a8c4f38edad4724ac733753692dda22`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:24:18.970030 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-308aa5d6-d97d-41ed-854b-5704bb7f1811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:19.048431 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1439/2873] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:24:19.064241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:24:19.066504 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:19.140875 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a965c-2164-4bd2-9a0f-c01042ba8b37 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:24:19.144678 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460407e1-96fe-4553-871b-9f8bce759ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:24:19.145343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460407e1-96fe-4553-871b-9f8bce759ffd service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:19.145625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460407e1-96fe-4553-871b-9f8bce759ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:19.145897 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460407e1-96fe-4553-871b-9f8bce759ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:19.156385 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460407e1-96fe-4553-871b-9f8bce759ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:24:19.157864 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460407e1-96fe-4553-871b-9f8bce759ffd service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:24:19.168079 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1435/2874] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:24:19.472390 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06628e6a-880f-457f-8188-1bb2c74a641e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:19.472689 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06628e6a-880f-457f-8188-1bb2c74a641e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:19.472971 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06628e6a-880f-457f-8188-1bb2c74a641e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:19.503715 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-06628e6a-880f-457f-8188-1bb2c74a641e None None] MFA Rules not processed for user `1a299663b56b411b9fc910ddf47103e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:24:19.505120 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-06628e6a-880f-457f-8188-1bb2c74a641e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:19.587555 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1440/2875] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:24:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:24:19.700444 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460407e1-96fe-4553-871b-9f8bce759ffd service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:24:19.701890 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-460407e1-96fe-4553-871b-9f8bce759ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:19.720963 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460407e1-96fe-4553-871b-9f8bce759ffd service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:24:19.725940 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:24:19.726939 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:19.727231 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:19.727551 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] PATH_INFO: `/v3/projects/26b009afec5a4e9d84281e6c8cf49c87` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:19.728805 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b009afec5a4e9d84281e6c8cf49c87)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:24:19.731094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:24:19.750090 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:24:19.788106 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b009afec5a4e9d84281e6c8cf49c87', 'request_id': 'req-a784a394-a4fa-4543-a3af-98ec706591e7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:19.796162 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b009afec5a4e9d84281e6c8cf49c87', 'request_id': 'req-a784a394-a4fa-4543-a3af-98ec706591e7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:19.804238 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b009afec5a4e9d84281e6c8cf49c87', 'request_id': 'req-a784a394-a4fa-4543-a3af-98ec706591e7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:19.820960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] The token cache is being invalidate because project 26b009afec5a4e9d84281e6c8cf49c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:24:19.821173 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:19.821960 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:24:19.824758 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1436/2876] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:24:19 2022] DELETE /identity/v3/projects/26b009afec5a4e9d84281e6c8cf49c87 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:24:25.341505 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06628e6a-880f-457f-8188-1bb2c74a641e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:24:25.342997 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-06628e6a-880f-457f-8188-1bb2c74a641e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:25.355521 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06628e6a-880f-457f-8188-1bb2c74a641e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:24:25.417743 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ab95d-96a6-4c6b-9967-e46d964d3ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:24:25.422547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ab95d-96a6-4c6b-9967-e46d964d3ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:25.422547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ab95d-96a6-4c6b-9967-e46d964d3ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:25.422547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ab95d-96a6-4c6b-9967-e46d964d3ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:25.422547 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b04ab95d-96a6-4c6b-9967-e46d964d3ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:25.476717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:24:25.496383 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ab95d-96a6-4c6b-9967-e46d964d3ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:24:25.497850 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ab95d-96a6-4c6b-9967-e46d964d3ef4 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:24:25.509534 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1441/2877] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:24:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 22 11:24:28.359394 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:24:28.360868 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:28.373365 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784a394-a4fa-4543-a3af-98ec706591e7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:24:28.376921 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b845b-37dc-467d-b928-e42e735a3806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:24:28.377557 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b845b-37dc-467d-b928-e42e735a3806 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:28.377872 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b845b-37dc-467d-b928-e42e735a3806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:28.378154 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b845b-37dc-467d-b928-e42e735a3806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:28.379894 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4e5b845b-37dc-467d-b928-e42e735a3806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:28.389771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b845b-37dc-467d-b928-e42e735a3806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:24:28.391194 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b845b-37dc-467d-b928-e42e735a3806 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:24:28.399279 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1437/2878] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:24:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:24:29.936962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ab95d-96a6-4c6b-9967-e46d964d3ef4 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:24:29.947561 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ab95d-96a6-4c6b-9967-e46d964d3ef4 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:24:29.951758 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:24:29.952402 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:29.952686 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:29.952962 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:29.954860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:29.969207 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:24:29.970681 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:24:29.981767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1442/2879] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:24:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:24:30.113470 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b845b-37dc-467d-b928-e42e735a3806 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:24:30.123161 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b845b-37dc-467d-b928-e42e735a3806 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:24:30.126849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:24:30.127518 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:30.127808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:30.128093 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:30.129874 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:30.195938 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:24:30.197368 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:24:30.207833 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1438/2880] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:24:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:24:39.976861 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=12 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:24:40.203466 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:24:43.500783 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:24:43.502792 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:43.524437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0308598-04c3-49d8-9a44-983fdc1151bb service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:24:43.531356 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:24:43.532162 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:24:43.532563 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:24:43.533045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:24:43.535841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:24:43.553437 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:24:43.555566 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:24:43.567617 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1443/2881] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:24:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:24:53.562709 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:25:12.955094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:25:12.956579 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:25:13.017019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:25:13.019319 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:25:13.034845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411d21c-0df9-4097-a29b-ada1ee99ebfa service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:25:13.040222 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd77dd8-49d4-4447-bfc5-91bd88792b9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:25:13.040876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd77dd8-49d4-4447-bfc5-91bd88792b9c service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:25:13.041278 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd77dd8-49d4-4447-bfc5-91bd88792b9c service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:25:13.041730 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd77dd8-49d4-4447-bfc5-91bd88792b9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:25:13.043663 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fcd77dd8-49d4-4447-bfc5-91bd88792b9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:25:13.055570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd77dd8-49d4-4447-bfc5-91bd88792b9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:25:13.057183 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd77dd8-49d4-4447-bfc5-91bd88792b9c service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:25:13.067174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1439/2882] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:25:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:25:13.086178 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459332b-e14e-4df3-9b55-37164b612f8b service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:25:13.091420 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:25:13.092330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:25:13.092615 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:25:13.092894 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] PATH_INFO: `/v3/users/00ab892a9233421886b849b879bd8a7e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:25:13.094112 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ab892a9233421886b849b879bd8a7e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:25:13.096052 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:25:13.141732 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ab892a9233421886b849b879bd8a7e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.149395 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ab892a9233421886b849b879bd8a7e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.156841 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ab892a9233421886b849b879bd8a7e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.168460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ab892a9233421886b849b879bd8a7e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.169091 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ab892a9233421886b849b879bd8a7e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.175110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1444/2883] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:25:13 2022] DELETE /identity/v3/users/00ab892a9233421886b849b879bd8a7e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 22 11:25:13.348648 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd77dd8-49d4-4447-bfc5-91bd88792b9c service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:25:13.360375 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd77dd8-49d4-4447-bfc5-91bd88792b9c service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:25:13.418494 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:25:13.419399 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:25:13.419692 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:25:13.419990 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] PATH_INFO: `/v3/projects/f756b95ab7da4271a50fa3b1a5276eea` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:25:13.421297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f756b95ab7da4271a50fa3b1a5276eea)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:25:13.423289 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:25:13.470188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f756b95ab7da4271a50fa3b1a5276eea', 'request_id': 'req-16808021-53c4-417b-865f-fda7dcd3926b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.476429 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f756b95ab7da4271a50fa3b1a5276eea', 'request_id': 'req-16808021-53c4-417b-865f-fda7dcd3926b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.482719 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f756b95ab7da4271a50fa3b1a5276eea', 'request_id': 'req-16808021-53c4-417b-865f-fda7dcd3926b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.496509 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] The token cache is being invalidate because project f756b95ab7da4271a50fa3b1a5276eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:25:13.496734 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.497355 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-16808021-53c4-417b-865f-fda7dcd3926b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:25:13.498909 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1440/2884] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:25:13 2022] DELETE /identity/v3/projects/f756b95ab7da4271a50fa3b1a5276eea => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:25:23.171455 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:25:23.492790 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:26:22.690349 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:26:22.691831 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:22.749712 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b4778-8bbb-44cc-9eaf-19fc1c9219e1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:26:22.797575 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b04cb4117974f8eb9bbf3a41c48a11a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:26:22.798256 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:26:22.798551 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:26:22.798830 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:26:22.800665 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:22.857436 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:26:22.858802 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:26:22.867988 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1445/2885] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:26:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:26:29.692343 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3916094-85fd-4e36-8c32-852a74f327da None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:26:29.692861 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3916094-85fd-4e36-8c32-852a74f327da None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:26:29.692973 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3916094-85fd-4e36-8c32-852a74f327da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:26:29.710941 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a3916094-85fd-4e36-8c32-852a74f327da None None] MFA Rules not processed for user `1a299663b56b411b9fc910ddf47103e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 22 11:26:29.712496 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a3916094-85fd-4e36-8c32-852a74f327da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:29.721068 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1441/2886] 10.222.0.20 () {58 vars in 957 bytes} [Fri Jul 22 11:26:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 22 11:26:30.035622 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:26:30.037053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:30.102806 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da3e1c-f9fb-48aa-b552-e28b9d7cb9d0 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:26:30.106366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:26:30.107013 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:26:30.107330 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:26:30.107611 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:26:30.109396 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:30.118943 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:26:30.120940 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:26:30.128852 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1446/2887] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:26:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:26:30.568620 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3916094-85fd-4e36-8c32-852a74f327da None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:26:30.570124 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a3916094-85fd-4e36-8c32-852a74f327da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:30.640849 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3916094-85fd-4e36-8c32-852a74f327da None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:26:30.645700 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e8fff-c913-4845-b16f-df5f1dbfb200 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:26:30.646346 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e8fff-c913-4845-b16f-df5f1dbfb200 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:26:30.646633 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e8fff-c913-4845-b16f-df5f1dbfb200 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:26:30.646917 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e8fff-c913-4845-b16f-df5f1dbfb200 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:26:30.648720 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d86e8fff-c913-4845-b16f-df5f1dbfb200 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:30.660439 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e8fff-c913-4845-b16f-df5f1dbfb200 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:26:30.662016 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e8fff-c913-4845-b16f-df5f1dbfb200 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:26:30.672767 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1442/2888] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:26:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:26:40.125266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:26:40.668348 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=35 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:26:42.916168 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:26:42.917590 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:43.120535 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f562c7-b26a-45bc-a814-283afba7164e service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:26:43.126026 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd739247-170f-478c-ab70-b68a2cce6f75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:26:43.126629 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd739247-170f-478c-ab70-b68a2cce6f75 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:26:43.126919 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd739247-170f-478c-ab70-b68a2cce6f75 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:26:43.127202 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd739247-170f-478c-ab70-b68a2cce6f75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:26:43.129015 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dd739247-170f-478c-ab70-b68a2cce6f75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:43.141538 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd739247-170f-478c-ab70-b68a2cce6f75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:26:43.143122 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd739247-170f-478c-ab70-b68a2cce6f75 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:26:43.154057 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1447/2889] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:26:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:26:50.723102 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e8fff-c913-4845-b16f-df5f1dbfb200 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:26:50.733927 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e8fff-c913-4845-b16f-df5f1dbfb200 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:26:50.737548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a41911-bf20-405b-9f1f-70c2a06f9b5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:26:50.738188 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a41911-bf20-405b-9f1f-70c2a06f9b5c service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:26:50.738479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a41911-bf20-405b-9f1f-70c2a06f9b5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:26:50.738760 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a41911-bf20-405b-9f1f-70c2a06f9b5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:26:50.740893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-53a41911-bf20-405b-9f1f-70c2a06f9b5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:50.752058 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a41911-bf20-405b-9f1f-70c2a06f9b5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:26:50.753534 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a41911-bf20-405b-9f1f-70c2a06f9b5c service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:26:50.761966 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1443/2890] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:26:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:26:51.126993 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd739247-170f-478c-ab70-b68a2cce6f75 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:26:51.136569 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd739247-170f-478c-ab70-b68a2cce6f75 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:26:51.140214 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:26:51.140858 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:26:51.141149 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:26:51.141546 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:26:51.143466 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:26:51.157814 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:26:51.159367 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:26:51.171158 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1448/2891] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:26:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:27:00.758360 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:27:01.165667 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=40 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:28:11.958584 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a41911-bf20-405b-9f1f-70c2a06f9b5c service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:11.970727 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a41911-bf20-405b-9f1f-70c2a06f9b5c service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:11.976531 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117febc-02f2-4ff1-9a05-3fed2f192b4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:11.977174 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117febc-02f2-4ff1-9a05-3fed2f192b4a service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:11.977464 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117febc-02f2-4ff1-9a05-3fed2f192b4a service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:11.977745 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117febc-02f2-4ff1-9a05-3fed2f192b4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:11.979558 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4117febc-02f2-4ff1-9a05-3fed2f192b4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:11.991907 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117febc-02f2-4ff1-9a05-3fed2f192b4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:11.993808 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117febc-02f2-4ff1-9a05-3fed2f192b4a service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:12.003682 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1444/2892] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:28:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:28:21.990989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:21.992602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:21.999546 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:28:22.008156 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6053cf9-0f3c-410b-b5b4-4ce331d5a380 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:22.014562 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c51929d0ecb047b4bdc405dc86fe7eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:22.015399 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:22.015696 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:22.016273 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:22.018081 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:22.033094 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:22.034710 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:22.048550 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1449/2893] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:28:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:28:28.030876 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117febc-02f2-4ff1-9a05-3fed2f192b4a service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:28.043114 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117febc-02f2-4ff1-9a05-3fed2f192b4a service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:28.048171 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecae842-8b8e-42a7-8a72-0df662c0a30f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:28.048835 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecae842-8b8e-42a7-8a72-0df662c0a30f service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:28.049123 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecae842-8b8e-42a7-8a72-0df662c0a30f service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:28.050147 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecae842-8b8e-42a7-8a72-0df662c0a30f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:28.052570 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cecae842-8b8e-42a7-8a72-0df662c0a30f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:28.065418 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecae842-8b8e-42a7-8a72-0df662c0a30f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:28.067149 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecae842-8b8e-42a7-8a72-0df662c0a30f service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:28.080251 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1445/2894] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:28:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:28:28.318853 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:28.320309 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:28.333200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbea620-72d7-41cc-a902-e924309dbb5c service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:28.336796 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:28.337447 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:28.337737 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:28.338019 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:28.339815 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:28.405160 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:28.406542 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:28.417152 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1450/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:28:29.530139 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecae842-8b8e-42a7-8a72-0df662c0a30f service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:29.540615 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecae842-8b8e-42a7-8a72-0df662c0a30f service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:29.544372 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffdf4c-e992-436f-93f7-7463372f4fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a0ce1804b514b42845ab33c7bfee309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:29.545032 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffdf4c-e992-436f-93f7-7463372f4fec service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:29.545323 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffdf4c-e992-436f-93f7-7463372f4fec service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:29.545827 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffdf4c-e992-436f-93f7-7463372f4fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:29.547732 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c9ffdf4c-e992-436f-93f7-7463372f4fec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:29.567608 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffdf4c-e992-436f-93f7-7463372f4fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:29.569096 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffdf4c-e992-436f-93f7-7463372f4fec service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:29.586981 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1446/2896] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:28:31.762446 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:31.764253 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:31.778205 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174728d-270e-4713-8744-aa6fd179768a service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:31.783011 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:31.784087 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:31.784532 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:31.784968 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] PATH_INFO: `/v3/users/11af2fb64a6c4a4ea660196aa90a9c9e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:31.789105 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=11af2fb64a6c4a4ea660196aa90a9c9e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:31.791045 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:31.867730 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11af2fb64a6c4a4ea660196aa90a9c9e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:31.877839 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11af2fb64a6c4a4ea660196aa90a9c9e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:31.888440 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11af2fb64a6c4a4ea660196aa90a9c9e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:31.907099 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11af2fb64a6c4a4ea660196aa90a9c9e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:31.907874 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11af2fb64a6c4a4ea660196aa90a9c9e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:31.916844 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1451/2897] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:28:31 2022] DELETE /identity/v3/users/11af2fb64a6c4a4ea660196aa90a9c9e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:28:32.127815 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffdf4c-e992-436f-93f7-7463372f4fec service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:32.141547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffdf4c-e992-436f-93f7-7463372f4fec service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:32.211605 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:32.212733 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:32.213025 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:32.213478 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] PATH_INFO: `/v3/projects/d084bf56f7894fb98a46d7e2cbb34157` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:32.214911 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d084bf56f7894fb98a46d7e2cbb34157)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:32.217092 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:32.283369 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd084bf56f7894fb98a46d7e2cbb34157', 'request_id': 'req-216873c6-a56d-4b07-b069-c50ffb39c5a8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.289771 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd084bf56f7894fb98a46d7e2cbb34157', 'request_id': 'req-216873c6-a56d-4b07-b069-c50ffb39c5a8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.296294 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd084bf56f7894fb98a46d7e2cbb34157', 'request_id': 'req-216873c6-a56d-4b07-b069-c50ffb39c5a8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.310984 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] The token cache is being invalidate because project d084bf56f7894fb98a46d7e2cbb34157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:28:32.311297 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.311929 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.313517 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1447/2898] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:28:32 2022] DELETE /identity/v3/projects/d084bf56f7894fb98a46d7e2cbb34157 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:28:32.318007 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:32.319684 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:32.333006 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e6c49-34b8-4268-b733-3babd40896bd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:32.383639 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:28:32.398855 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:32.399795 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:32.400084 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:32.400366 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] PATH_INFO: `/v3/users/b460166917df4ee49cb59f942fed5a51` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:32.401648 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b460166917df4ee49cb59f942fed5a51)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:32.403776 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:32.461311 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b460166917df4ee49cb59f942fed5a51'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.469739 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b460166917df4ee49cb59f942fed5a51'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.478392 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b460166917df4ee49cb59f942fed5a51'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.492164 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b460166917df4ee49cb59f942fed5a51'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.492873 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b460166917df4ee49cb59f942fed5a51'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.499067 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1452/2899] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:28:32 2022] DELETE /identity/v3/users/b460166917df4ee49cb59f942fed5a51 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:28:32.676391 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:32.677851 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:32.696565 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216873c6-a56d-4b07-b069-c50ffb39c5a8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:32.755978 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:32.756854 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:32.757260 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:32.757548 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] PATH_INFO: `/v3/projects/efb23709e5fc4d6ea06087c45b13f0e9` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:32.758736 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb23709e5fc4d6ea06087c45b13f0e9)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:32.760718 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:32.767094 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:28:32.816749 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb23709e5fc4d6ea06087c45b13f0e9', 'request_id': 'req-c892c730-4d9e-4599-9645-ad4c7c4e6d22'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.822080 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb23709e5fc4d6ea06087c45b13f0e9', 'request_id': 'req-c892c730-4d9e-4599-9645-ad4c7c4e6d22'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.828716 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb23709e5fc4d6ea06087c45b13f0e9', 'request_id': 'req-c892c730-4d9e-4599-9645-ad4c7c4e6d22'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.841699 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] The token cache is being invalidate because project efb23709e5fc4d6ea06087c45b13f0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:28:32.841995 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.842626 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:32.844212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1448/2900] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:28:32 2022] DELETE /identity/v3/projects/efb23709e5fc4d6ea06087c45b13f0e9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:28:35.608132 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:35.609609 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:35.623186 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9d232-65c3-4341-813e-93d5b003e6f7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:35.683184 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:35.684213 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:35.684583 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:35.684862 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] PATH_INFO: `/v3/users/1a299663b56b411b9fc910ddf47103e5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:35.686053 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a299663b56b411b9fc910ddf47103e5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:35.688022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:35.738621 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a299663b56b411b9fc910ddf47103e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:35.746887 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a299663b56b411b9fc910ddf47103e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:35.754222 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a299663b56b411b9fc910ddf47103e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:35.761860 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:28:35.766976 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a299663b56b411b9fc910ddf47103e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:35.767767 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a299663b56b411b9fc910ddf47103e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:35.775167 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1453/2901] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:28:35 2022] DELETE /identity/v3/users/1a299663b56b411b9fc910ddf47103e5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:28:35.946034 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:35.947547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:35.960893 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892c730-4d9e-4599-9645-ad4c7c4e6d22 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:36.018831 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:36.020259 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:36.020547 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:36.020828 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] PATH_INFO: `/v3/projects/04d58bceb8414b02a1bbaec0db07d85e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:36.022011 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d58bceb8414b02a1bbaec0db07d85e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:36.023997 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:36.069137 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d58bceb8414b02a1bbaec0db07d85e', 'request_id': 'req-7ee3944f-22f0-4428-bed6-b2fea556ad0e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.074528 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d58bceb8414b02a1bbaec0db07d85e', 'request_id': 'req-7ee3944f-22f0-4428-bed6-b2fea556ad0e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.080479 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d58bceb8414b02a1bbaec0db07d85e', 'request_id': 'req-7ee3944f-22f0-4428-bed6-b2fea556ad0e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.092951 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] The token cache is being invalidate because project 04d58bceb8414b02a1bbaec0db07d85e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:28:36.093241 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.093867 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.095625 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1449/2902] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:28:35 2022] DELETE /identity/v3/projects/04d58bceb8414b02a1bbaec0db07d85e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:28:36.099679 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:36.101106 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:36.113823 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bfa9a-ce96-4c9f-9320-613af73ca66a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:36.168422 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:36.169220 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:36.169507 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:36.169786 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] PATH_INFO: `/v3/users/e253928c11e64bb5b187f4f3dd8cf95f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:36.170956 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e253928c11e64bb5b187f4f3dd8cf95f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:36.172911 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:36.219506 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e253928c11e64bb5b187f4f3dd8cf95f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.227039 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e253928c11e64bb5b187f4f3dd8cf95f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.233785 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e253928c11e64bb5b187f4f3dd8cf95f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.245746 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e253928c11e64bb5b187f4f3dd8cf95f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.246377 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e253928c11e64bb5b187f4f3dd8cf95f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.252200 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1454/2903] 10.222.0.20 () {58 vars in 1223 bytes} [Fri Jul 22 11:28:36 2022] DELETE /identity/v3/users/e253928c11e64bb5b187f4f3dd8cf95f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:28:36.439564 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:28:36.441649 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:28:36.454438 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3944f-22f0-4428-bed6-b2fea556ad0e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:28:36.475132 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:28:36.514645 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8c4f38edad4724ac733753692dda22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25f160b71984d6fbacad5d6c1f3248c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:28:36.515568 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:28:36.515855 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:28:36.516133 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] PATH_INFO: `/v3/projects/4ed3871a9ef44759abf5a216c756e1df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:28:36.517298 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed3871a9ef44759abf5a216c756e1df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:28:36.519283 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:28:36.563600 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed3871a9ef44759abf5a216c756e1df', 'request_id': 'req-7f2e3cc3-b8db-404b-895c-c5131b8113b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.569744 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed3871a9ef44759abf5a216c756e1df', 'request_id': 'req-7f2e3cc3-b8db-404b-895c-c5131b8113b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.575711 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed3871a9ef44759abf5a216c756e1df', 'request_id': 'req-7f2e3cc3-b8db-404b-895c-c5131b8113b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.598845 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] The token cache is being invalidate because project 4ed3871a9ef44759abf5a216c756e1df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 22 11:28:36.599145 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.599977 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7f2e3cc3-b8db-404b-895c-c5131b8113b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 22 11:28:36.601656 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1450/2904] 10.222.0.20 () {58 vars in 1232 bytes} [Fri Jul 22 11:28:36 2022] DELETE /identity/v3/projects/4ed3871a9ef44759abf5a216c756e1df => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 22 11:28:46.248717 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=56,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:28:46.585212 dv-cinder-848900-4-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 22 11:33:17.922266 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 22 11:33:17.923727 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:33:18.061989 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2b58d-c417-453c-bbff-d74091a29e46 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 22 11:33:18.120171 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022a843-29bf-471b-85ae-b0169f09653d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88ecd4c645174d28930a44b93cfee9b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89224d740c7148938c2d803804083a57', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 22 11:33:18.120818 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a843-29bf-471b-85ae-b0169f09653d service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 22 11:33:18.121110 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a843-29bf-471b-85ae-b0169f09653d service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 22 11:33:18.121656 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a843-29bf-471b-85ae-b0169f09653d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 22 11:33:18.123460 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6022a843-29bf-471b-85ae-b0169f09653d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 22 11:33:18.135126 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022a843-29bf-471b-85ae-b0169f09653d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 22 11:33:18.138022 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022a843-29bf-471b-85ae-b0169f09653d service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 22 11:33:18.148602 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1455/2905] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Jul 22 11:33:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 22 11:33:28.144299 dv-cinder-848900-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}